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Executive Summary 


Context for the Study: Education Development Center’s 


Multi-Part Evaluation 

As part of the Ready to Learn Initiative, Education Development Center, Inc. (EDC), was 
charged with addressing the evaluation of Technological Enhancements for the outreach 
efforts of three producers: Out of the Blue’s Super WHY! Technology Add-On; Sesame 
Workshop’s The Electric Company School’s Initiative Curriculum; and WordWorld’s 
eBook Library. 


Overview 

The report presents information about each of the properties, provides logic models 
describing each program’s components and expected outcomes, describes evaluation 
methods for data collection and analysis, discusses findings for individual properties, 
presents a discussion of findings across properties, and finally offers recommendations 
to further the use of technological enhancements to increase the reach of media 
products targeting early literacy skills. 


Methods: 


Researchers collected data from multiple sources to inform the evaluation, including 
print and Web-based documents; interviews with teachers, parents, and property 
producers; site visits that included observations of adults and children interacting with 
each evaluated property; and online surveys. 


Qualitative data analysis methods were used to review, code and analyze the range of 
data collected. Print and web-based material was reviewed and catalogued; interview 
data were transcribed, entered into Atlas.ti, a qualitative data analysis tool, and coded; 
observations were reviewed and analyzed for common themes as well as property- 
specific themes; and survey data were analyzed through generation of response 
frequencies and review and coding of responses to open-ended questions. 


Research Questions 


The evaluation addressed three central research questions: 

* Do the supplemental materials or activities address a gap or missing link in the 
overall dissemination and communication plan of the product, filling a void that 
would otherwise go unmet by existing products and services? 

¢ Do the supplemental materials meet the needs of the end user (caregiver, 
teacher, or child) in a way that provides a novel/innovative/valued experience? 


* Do the supplemental materials alter the interactions between caregivers and 
young children with respect to early literacy experiences? 


Technological Enhancements 


Out of the Blue’s Super WHY! Technology Add-On 

Out of the Blue Enterprises developed an online games extension that built on their 
existing Super WHY! Summer Reading Camp. The camps provide access to literacy- 
oriented games, hands-on activities, video viewing that is linked to early literacy skill 
activities, and computer games that reinforce early literacy skills. The computer games 
introduce practice targeting a different early reading skill each day. Each game focuses 
on the particular literacy “power” possessed by one of the four Super WHY! Characters 
including: alphabet power, spelling power, word power, and reading power. 


Overall, responses from camp facilitators were positive with regard to the content, 
design, and implementation of the Super WHY! Summer Reading Camp with technology 
add-on. The program was described as engaging, fun, colorful and stimulating for 
campers, and also offered a chance to engage with technology. Some facilitators stated 
that the program included games that were too challenging for their campers and 

others suggested that games requiring more basic early literacy skills were more popular 
choices for participating children. 


Recommendations for this property include: 

¢ Provide a digital tour of the curriculum materials so that teachers can have access, 
on their own time, to a guided introduction to the program, its characters and the 
underlying literacy goals. 

¢ Provide teachers with a set of the Super WHY! games on a CD so that teachers do 
not have to rely on internet access to fully implement the curriculum. 


Sesame Workshop’s The Electric Company School’s Initiative Curriculum 

Sesame Workshop’s The Electric Company partnered with Success for All to develop a 
hands-on curriculum using whiteboard technology. The result is a curriculum that allows 
teachers to deliver Success for All phoneme lessons using videos and games that borrow 
characters from The Electric Company story line. 


Teachers were generally very satisfied with The Electric Company School’s Initiative 
Curriculum, reporting that it was engaging, fun, and encouraged student participation in 
activities that are otherwise unappealing or less interesting. Some teachers indicated 
that the pacing of lessons could be faster, and others indicated that there was not 
always a good match between student grade level and content, and between the 
Success for All lesson and The Electric Company School’s Initiative Curriculum. 


Recommendations for this property include: 


¢ Review pacing of the games, which was too slow for some teachers, and also the 
overall time requirement of 45 minutes for full implementation. Both the game time 
and overall program time could be shortened. 

¢ Provide more opportunities for interaction during the games, and have games 
reflect student interaction, such as providing different scores based on different 
playing levels. 


WordWorld’s eBook Library 

WordWorld’s eBook is an electronic book that provides audio and animation as part of 
the book reading experience. When viewed the eBook will “read” the text out loud, 
highlight each word as it is read, and encourage page exploration through animations 
that demonstrate how letters come together to form words. 


Parents were positive about their experience reading an eBook with their child, and 
indicated that the pacing of the highlighted words, the visual images and the animations 
were all positive qualities of the book. Children also enjoyed and were engaged by the 
eBook. Parents who were Spanish speaking indicated they would like to know what the 
story is about and would benefit from having the story text available in Spanish, which 
would allow them to help their children better understand the English words 
incorporated into the book. 


Recommendations for this property include: 

¢ Provide additional common elements of book design by including a title page, 
author and illustrator to support children’s exposure to these aspects of book 
reading. 

¢ Provide a Caregiver Guide that is more simplified and offers suggestions for 
caregivers that can be implemented individually or as a series of activities. 

¢ Provide a Spanish language version of the Caregiver Guide. 

¢ Provide the book’s text in Spanish to support parent’s understanding of the story so 
that they can help their children with comprehension. 


Discussion and General Recommendations 


Discussion 

Despite the substantial differences in the goals and methods of the three supplemental 
technological enhancement efforts, some elements are common to all three. The most 
prominent of these is the focus producers had on supporting literacy skills for 
underserved children and the adults who care for them, and the use of technology to 
reach out to these communities in innovative ways. Although the producers used 
unique technologies to address different child and adult populations (from age 3 
through 8; and teachers or parents), all sought to ensure that every child had access to 
experiences that would support early literacy learning. All three also took advantage of 
current and innovative technology to support this goal. If we return to the three central 


evaluation questions when considering the outcome of all three properties, we must 
consider some other common findings. Our analysis of data from each property indicate 
that each property produced positive outcomes in response to one or more of the 
study’s three central research questions. 


Question one asked whether the supplemental materials or activities address a gap or 
missing link in dissemination and communication that would otherwise go unmet by 
existing products and services. Of the three properties, two appeared to meet this need: 
the Super WHY! Summer Reading Camp with technology add-on and The Electric 
Company School’s Initiative Curriculum both address specific gaps raised by teachers 
and identified by producers as they sought to develop their technological 
enhancements. There was no evidence of a gap or need filled by the development of an 
eBook Library CD; however, it should be noted that parents said they thought having an 
eBook on a CD would encourage a child to use it more often because the simple act of 
seeing the CD would be a reminder. 


Question two asked whether the supplemental materials meet the needs of the end 
user in a way that provides a novel experience. All three properties were able to address 
this issue in ways that reflect their individual goals. The Super WHY! Summer Reading 
Camp games provided young children with engaging experiences that supported 
practice of early reading skills. Although not all games were considered academically 
appropriate for all children, camp facilitators were able to support their children’s use of 
at least two of the four games and indicated that children benefited from the 
experience and would not have had the same experience without the games. The 
Electric Company School’s Initiative Curriculum exposed students to phoneme practice 
in ways that were engaging, visual, and fun, and provided some relief from what was 
perceived as an otherwise less visually interesting curriculum. Lastly, the WordWorld 
eBook provided children and their caregivers with a reading experience that could be 
modified to meet their needs (with or without reading aloud) and encouraged parent 
engagement in the early reading experiences of young children. 


Question three asked whether the supplemental materials alter the interactions 
between caregivers and young children with respect to early literacy experiences. In this 
case we suggest that only the eBook provided some evidence of altering interactions 
between a child and caregiver because of the engaging animations, story line, and visual 
and auditory information about letters, sounds, and the words they make. However, we 
also suggest that the other properties have the potential to alter informal interactions 
between adults and students because of their engaging nature and their use of humor, 
physical activity, and singing to encourage participation. Both The Electric 
Company/Success for All video segments and the games included in the Super WHY! 
Summer Reading Camp seek to engage children in age-appropriate activities such as 
singing, or game-play in ways that reinforce a specific concept (such as the alphabet or a 
phonic element). Once children are engaged they are more focused on a task, more 
likely to retain the information that is presented and are more likely to receive positive 


reinforcement from a teacher (who may not provide as much positive reinforcement in 
a setting where children are not engaged or focused). In this way the technological 
enhancements may support altered interactions between adults and children in 
different settings. 


Exploring innovative uses of technology to achieve the goals laid out by the three 
properties included in this study is a highly challenging undertaking, but it also can be a 
rewarding process. Technology has provided a broad range of new approaches to 
reaching out to children, parents, and educators, but exploring which of these 
approaches is best suited to a particular audience or task will require a considerable 
amount of experimentation. Efforts such as the Technological Enhancements project will 
provide guidance and leadership for future work in this area. 


Recommendations 

Ongoing support for the development of programs and outreach efforts that include 
digital tools requires a willingness to experiment with new and existing technologies, 
and with innovative approaches to integrating media into learning experiences for 
young children and their care-providers. Undertaking such a development process may 
be out of the ordinary for many traditional public media producers. Yet fostering an 
environment where this kind of development is encouraged and supported must take 
place if new ways of reaching new and existing audiences are to be discovered. We 
recommend that CPB and its partners continue supporting this kind of innovation and 
exploration by integrating the following into their current and future programs: 


¢ Support communication across producers and public media stations 
experimenting with digital tools in a range of learning contexts. In this way 
different properties may be able to build on the experiences of others, which 
may lead to opportunities for collaboration across properties and will contribute 
to a shared culture of innovation and iterative development. 


* Conduct ongoing needs assessments of target audiences. Although technology 
may offer innovative ways to reach out to those whom public media is seeking to 
support (e.g., families, teachers, and learners), access to different kinds of 
technology is still limited for some populations. Ongoing needs assessment can 
help producers and stations determine which audiences have access to which 
technologies, and which audiences are benefiting from newly introduced 
technological enhancements. 


* Recognize that developing new and innovative uses of technology requires room 
for experimentation and for mistakes. Providing opportunities for producers to 
experiment and to recognize and learn from mistakes may help to create an 
environment where innovation leads to new ways of reaching out to children 
who are most in need of support. 
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Introduction 


This report contains findings of the evaluation of the Technological Enhancements 
project conducted by Education Development Center, Inc. (EDC), for the Corporation for 
Public Broadcasting (CPB). The study examines the role that added technology in the 
form of whiteboards, computer games, and electronic books has played in increasing 
the outreach efforts and early learning impact of three properties working under the 
Ready to Learn umbrella of programming. Each of the three properties— Out of the 
Blue’s Super WHY! Technology Add-On, Sesame Workshop’s The Electric Company 
School’s Initiative Curriculum and WordWorld’s eBook Library —builds on the potential 
of technology in different ways, yet all seek to further the early literacy learning 
experiences of young children and engage digital tools to accomplish this goal. 


In fall 2009, CPB sought to further the outreach efforts of some of its grantees that were 
well positioned to build on existing programming using technology. Three producers 
were selected to receive supplemental funding for the development of technological 
enhancements that would further the reach of their existing programs and materials. 
These grantees began modification and development of their products in late 2009, and 
rolled out their products to target audiences beginning in May of 2010, with the final 
roll-out taking place in September of 2010. Our evaluation coincided with the 
development activities. Scheduling of start dates for technological enhancements was 
delayed by some months for each of the properties; however, researchers included all 
properties in the evaluation and worked with individual producers to accommodate 
shifts in scheduled implementation. 


CPB and its partners, including Out of the Blue Enterprises, Sesame Workshop, and 
WordWorld, have demonstrated a commitment to developing programming that 
addresses the educational needs of children from traditionally underserved 
communities. In recent years, through support from the Ready to Learn Initiative, these 
producers have developed and delivered programming to children, parents, and 
educators that targets a range of early learning skills identified by the National Reading 
Panel (2000) as central to learning how to read. In doing so they have built on current 
and new technologies and have drawn on recent research about the skill components 
necessary for reading acquisition. The Supplemental Outreach funding provided added 
resources for producers to further explore the possibilities of reaching out to target 
audiences, or reaching further to extend depth of knowledge or experience with a 
target skill or task. 


Educators are acutely aware of the challenges inherent in reaching underserved and 
academically at-risk populations, yet they are also aware of the consequences of not 


reaching out and exploring every avenue of access available to make this effort 
successful. 


A substantial amount of research has demonstrated that acquiring early literacy skills is 
a crucial step in the life of a young child. Children who are not proficient in their early 
elementary grades in such key early skills as letter naming, identifying letter sounds, and 
understanding basic concepts of print (Lepola, Poskiparta, Laakkonen, & Niemi, 2005; 
Roth, Speece, & Cooper, 2002) and reading comprehension (Kendeou, van den Broek, 
White, & Lynch, 2009; Storch & Whitehurst, 2002; Whitehurst & Lonigan, 1998) struggle 
in later grades as readers. They often do not catch up to their peers (Juel, 1988; 
Torgesen, 2004) and are more likely to experience reading difficulties as they progress 
through school (Snow, Burns, & Griffin, 1998). Reaching out to these children early and 
often may make the difference between academic struggle and academic success. 
Identifying alternate pathways to reach all students with engaging content that supports 
literacy learning has been a challenge to educators across the country. 


Technology often has been identified as a viable tool to support learning in general and 
literacy learning in particular, and several studies have noted its potential to support 
learning among young children (Linebarger, Taylor-Piotrowski, & Vaala, 2007) and to 
support targeted early literacy skillls in particular (Penuell et al., 2009). Digital media can 
increase student engagement (Clements, 2002; Clements, Nastasi, & Swaminathan, 
1993), may increase attention to narrative (Crawley et al., 2002), and can support social 
interactions when engaged in jointly (Dugan, Stevens, & Mehus, 2010). Digital media 
show particular promise when they are engaged within a setting that includes adult or 
peer guidance (Collins, Sobol, & Westby, 1981; Lemish & Rice, 1986; Linebarger & 
Walker, 2005; Nathanson, 2001a, 2001b; Reiser, Tessmer, & Phelps, 1984; Reiser, 
Williamson, & Suzuki, 1988; St. Peters, Huston, & Wright, 1989). 


The three programs identified in this report all build on these findings and extend them 
to different student and adult populations. They also draw on different elements of 
available digital tools to achieve a particular goal. The potential for technology to 
transform and extend the reach, with respect to depth and breadth, is explored in these 
three programs and provides an opportunity for educators to push the boundaries of 
common practice in supporting young children as they develop essential literacy skills. 


This report describes the goals of each property with regard to its added technological 
components, and then addresses three central research questions: 

1. Do the supplemental materials or activities address a gap or missing link in the 
overall dissemination and communication plan of the product, filling a void that 
would otherwise go unmet by existing products and services? 

2. Do the supplemental materials meet the needs of the end user (caregiver, 
teacher, or child) in a way that provides a novel/innovative/valued experience? 

3. Do the supplemental materials alter the interactions between caregivers and 
young children with respect to early literacy experiences? 
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We have organized findings in this report so that they are first discussed for each 
property. We then discuss cross-property themes. The final section offers 
recommendations for individual properties as well as overall programming with regard 
to the role of technology to support outreach, and early learning goals. 


Program Descriptions 

Below we provide program descriptions of each property in this study. These 
descriptions were drawn from print and Web-based documents collected by research 
staff, and provided by producers and their staff from Out of the Blue, The Electric 
Company, and WordWorld. These descriptions provide a context for the findings 
presented later and also helped to inform the development of interview, observation, 
and survey protocols. In addition to preparing these descriptions, researchers developed 
logic models for each property based on materials and information available. We 
updated the models with new information as data were collected to present an external 
view of each program’s inputs, activities, and intended outcomes. 


Out of the Blue’s Super WHY! Summer Reading Camp Technology Add-On 


Why, Super WHY! character, drawn by a Volunteer Universal Pre-Kindergarten student in 
Florida. 


Out of the Blue Enterprises developed an online games extension that built on its 
existing Super WHY! Summer Reading Camp. The camps target children age 4 to 5 who 
attend a weeklong summer session and provide access to literacy-oriented games, 
hands-on activities, video viewing that is linked to early literacy skill activities, and 
computer games that reinforce early literacy skills. 
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The camp facilitators are educators who are classroom teachers during the school year. 
Camp facilitators receive detailed curriculum materials that lay out the components of 
the camp day, the materials needed to complete activities, and the language they can 
use to introduce the video segments, computer games, and hands-on activities to their 
campers. 


Background 


Before it received funding for technological enhancements, Super WHY! Summer 
Reading Camp included all elements of the current program except for inclusion of 
computer games: video clips, hands-on activities, and games for children that built on 
themes addressed in the video segments. 


Description of the Technological Enhancement 


Out of the Blue worked with a pre-kindergarten teacher to develop a modified Super 
WHY! Summer Reading Camp curriculum that incorporated five Super WHY! computer 
games in ways that support target skills introduced through Super WHY! characters. All 
available on the PBS KIDS website, one Super WHY! computer game was incorporated 
into each day’s activity plan, with a camp facilitator either guiding the mouse while a 
group of children play, or children playing the game by themselves in groups of two or 
three. Children spend approximately 10 minutes each day playing these computer 
games. 


Target Setting and Literacy Topics Addressed 


The Super WHY! Summer Reading Camp with Technology Add-On program is designed 
for implementation in a classroom setting for pre-kindergarten children. Classrooms 
must have access to a computer that has Internet access and a sound card and speakers 
so that children can play the program’s computer games, and a video monitor so they 
can watch Super WHY! episodes as a group activity. In addition, the program requires 
that camp facilitators have access to a printer (for printing materials such as bingo 
cards), and to basic art and craft materials to support the activities in the program’s 
curriculum document. 


The Super WHY! Summer Reading Camp program builds on a single Super WHY! episode 
which is introduced and viewed each day. The episode introduces each of the four 
Super WHY! characters and the reading skill that they possess: letter identification; 
rhyming and decoding; spelling and encoding; and reading and comprehension. In the 
episode the characters attempt to use these different skills to solve a problem. The 
computer games introduced each day target one of these four skills; for example, on 
Day 2, campers practice rhyming with Wonder Red’s Freeze Dance Rhyming game and 
practice making word families. The Camp Facilitators guide outlines specific skills: for 
example, identifying letters such as W, O, L, F and making a word family with “all” 
including Ball, Call, Hall, etc. 
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Dissemination: User Access to the Product 


Public television stations disseminate the Super WHY! Summer Reading Camp program. 
Within each station an education or outreach coordinator makes contact with local 
schools and offers the program to them. For example, in one location the outreach 
coordinator established a relationship with a local Title | coordinator and this resulted in 
the participation of multiple classrooms. Eighty-six classrooms across the country 
implemented the Super WHY! Summer Reading Camp with Technology Add-On program 
in the summer of 2010. 


Product Training 


There is no specified training for this program; materials are designed to support 
facilitator implementation without the need of any additional guidance. The program 
provides a highly detailed curriculum guide that is scripted and offers facilitators a clear 
schedule integrating all activities throughout the day. These materials are designed to 
provide a complete implementation package that any camp facilitator could implement 
regardless of background or experience. 


Target Audience: Description of End Users 


The target audience for the Super WHY! Summer Reading Camp with Technology Add- 
On is pre-kindergarten students living in low-income households. One document 
provided by the program’s producer indicates that campers should be school-ready pre- 
kindergarten students. Settings must also have access to functioning technology (as 
mentioned above) and must include facilitators who are somewhat familiar with 
computer games for children or are willing to try using games in a classroom setting. The 
camp program also requires some time and funding resources to prepare a classroom 
for the weeklong session. 


Below we show a logic model of the program’s inputs, activities, outputs and intended 
short- and long-term outcomes (see Figure 1). 
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Figure 1: Logic model for Super WHY! Summer Reading Camp with Technology Add-On. 


Out of the Blue Enterprises’ Technology Enhanced Super WHY! Summer Reading Camp 
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Sesame Workshop’s The Electric Company School’s Initiative Curriculum 
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Screen shot from The Electric Company website. 


Sesame Workshop’s The Electric Company (TEC) partnered with Success for All (SFA) to 
develop a hands-on curriculum using whiteboard technology. The result is a set of 
curricula that allows teachers to deliver SFA phoneme lessons using videos and games 
that borrow characters from TEC story line. 


Target Setting and Literacy Topics Addressed 


The Electric Company School’s Initiative Curriculum uses SFA’s Fast Track Phonics 
lessons infused with TEC content, and allows teachers to integrate media and games as 
they build students’ skills in letter-sound correspondence (phonemic awareness) and 
word-level reading and spelling. The target setting for TEC School’s Initiative Curriculum 
is a classroom equipped with interactive whiteboard technology where students (grades 
1 and 2) are being taught phoneme concepts. The outreach model provides curriculum 
materials to 24 teachers who have been trained in the Fast Track Phonics program. 


Dissemination: User Access to the Product 


In participating classrooms TEC School’s Initiative Curriculum was distributed on a CD 
and installed on the teacher’s computer. The computer, desktop or laptop, was 
connected to the classroom’s projector and interactive whiteboard, such as a 
Promethean or Smartboard product. The interactive whiteboard technology allows 
teachers to interact with media by viewing videos or playing games. Although the 
interactive whiteboard technology is the intended platform for interacting with and 
delivering the TEC/SFA content, teachers can also use the computer’s mouse to execute 
the content without touching the whiteboard interface. The whiteboard technology 
allows for the presentation of material to an entire classroom full of students and 
supports large-group instruction more than the use of single computers. TEC School’s 
Initiative Curriculum does not rely on the Internet; rather lessons are made accessible 
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online for teachers in case of problems with the program’s CD. This option also allows 
teachers to access the material from home for planning and lesson preparation. 


Product Training 


All participating teachers receive training in the use of TEC School’s Initiative Curriculum 
from an SFA reading coach with the support of a Sesame Workshop curriculum 
specialist. Trainings generally take place in classrooms equipped with a laptop, 
projector, and interactive whiteboard technology. The trainings highlight the alignment 
between TEC videos, games, and SFA lessons and include modeling good practices for 
teachers, such as how to organize game play and manage group viewing of TEC videos. 


A goal of the training is to provide teachers with instructional support in structuring and 
scheduling their Fast Track Phonics lesson plans with TEC School’s Initiative Curriculum, 
which requires 45 minutes for full implementation of the day’s lesson. Generally, all SFA 
schools are required to implement a 90-minute literacy period, and TEC producers 
expect TEC School’s Initiative Curriculum to be delivered during that time. 


Target Audience: Description of End Users 


The end users of TEC School’s Initiative Curriculum fall into two groups: teachers and 
students. Teachers who implement TEC School’s Initiative Curriculum should be 
experienced users of the SFA program and familiar with teaching literacy during the 
daily 90-minute literacy block. These teachers will deliver TEC School’s Initiative 
Curriculum to their students during this period for 45 minutes. 


Additional Resources 


Sesame Workshop also developed the Digital Literacy Resource (DLR), an online 
curriculum database made available through an independent link provided to teachers. 
Teachers locate lesson plans, depending on their literacy interests and access to digital 
and print materials, to use as an accompaniment to their lessons or as additional 
homework activities. Although the DLR contains all the material from TEC School’s 
Initiative Curriculum, as well as additional literacy tools, it is presented in a database 
format rather than the more interactive format used for TEC School’s Initiative 
Curriculum. 


Below we show a logic model of the curriculum’s inputs, activities, outputs, and 
intended short- and long-term outcomes (see Figure 2). 
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Logic Model: Figure 2, The Electric Company / Success for All 


Sesame Workshop's The Electric Company School's Initiative Curriculum 
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WordWorld’s eBook Library 


It was picture day in WordWorld, and 
Duck wanted tosmake sure he looked 
TOTALLYiferrific: 

aw 


Screen shot from the Totally Terrific Duck eBook on the WordWorld website. 


WordWorld’s eBook is an electronic book that provides audio and animation as part of 
the book reading experience. When viewed the eBook will “read” the text out loud, 
highlight each word as it is read, and encourage page exploration through animations 
that demonstrate letters coming together to form words. 


Background 


WordWorld is an animated children’s television show on PBS. The show is set in a three- 
dimensional world full of letters and words, where characters called WordFriends have 
adventures while solving problems. In each episode WordFriends look for letters and 
spell words. Letters “morph” together to depict their meaning. For example, the letters 
M-U-G in the word MUG will push together into the shape of an actual mug. This type of 
“word building” is the main educational focus of the program, as it illustrates the 
connection between letters, words, and meaning. The objects, scenery items, and 
characters in the show are all made from the letters that spell them. Currently five 
eBooks are available on the WordWorld website: www.wordworld.com. 


Description of the Technological Enhancement 


The eBook Library technological enhancements and outreach plan aims to provide an 
enhanced format that presents the pedagogical goals of the television series more 
interactively and on additional platforms such as CD ROMs and smart phones. 
Producers intended the new eBook to be a richer overall experience for the user than 
the original eBook format. The new book is longer and follows a more in-depth story 
line. It aims to offer the reader a more interactive and playful experience, with 
expanded options for animation and word-building opportunities, audio, and text 
narration. The new eBooks include additions to the interactive activities and games that 
the original eBooks introduced. Enhanced visual graphics give the user a more engaging 
and exciting experience. Additional elements will make the new eBook easier for young 
children to navigate through and read. For example, when the user scrolls over a “hot 
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spot” on the screen, the pointing finger is enlarged, signaling to select that spot to see 
an animation. 


Target Setting and Literacy Topics Addressed 


The eBook Library stories are intended to foster a love of language, words, and story 
telling in children by making the reading experience more fun. The eBook Library stories 
have silly characters, animations that involve letters and words, and rhymes that help 
children make connections and build an understanding of letters, sounds, and words. 
The educational goals of the books focus on important pre-literacy skills identified by 
the National Reading Panel. In addition to the word-building focus the eBook Library 
pinpoints vocabulary, phonics, and phonemic awareness skills. One strategy the books 
use to reach these goals is repetition. Throughout a book, words, phonemes, and 
sounds are repeated to help the young user internalize these new skills. The books also 
introduce rhymes to enhance phonemic awareness and increase reading success. 


Users can click “hot spots” throughout the books to hear a word read out loud or see an 
animation. All text on the page is a hot spot, as well as some pictures. If a child chooses 
to read the book independently, without narration, he or she may click any unfamiliar 
word to hear the word read out loud. A child may also click on certain objects or 
characters to initiate animation. In addition, the words on the screen are highlighted as 
they are read, supporting important early reading skills like one-to-one correspondence 
and tracking. 


Woven into an eBook are opportunities for children to interact with the story. When 
doing these activities the child chooses the letters to “build a word” that is in the story. 
Children use a mouse to click on letters spread around the page of the eBook. If a child 
does not click on a letter within a few seconds, the letters will automatically move to 
form the word. Once one letter is chosen, the others “pulse” on the screen making them 
more visible to the young user. When the word is completely formed the narrator 
pronounces the word slowly, enunciating each of the different sounds in the word. 


At the end of the eBook another interactive game reviews a skill introduced in the story. 


Dissemination: User Access to the Product 


The eBook Library will be disseminated to educators, children, and caregivers through 
different platforms. They will be available online at websites www.wordworld.com and 
www.pbskids.org, where they can be accessed on demand by a single click and read 
over and over again. The eBook Library and caregiver guides will be distributed on a CD, 
with help from a variety of partner organizations such as First Book and Head Start, 
which serve low-income children and families. And children and caregivers will receive 
the CDs at home through HIPPY (Home Instruction for Parents of Preschool Youngsters) 
and the Parent-Child Home Program, both of which send service providers directly into 
homes to work with children and parents. Lastly, an iPhone/iPad application is in 


19 


development and will allow the interactive books to be downloaded and read ona 
mobile, hand-held device such as a smartphone. 


Product Training 


Each eBook includes a Caregiver Guide that highlights vocabulary words and letters 
introduced in the story; suggests possible questions to ask a child before, during, and 
after reading the story; and provides extension activities and ideas for adults to help 
children make connections between the text and their own lives. 


Target Audience: Description of End Users 


Like the WordWorld television series, the eBook was developed for an audience of 
three- to six-year-old children. A child can choose to read the eBook with an adult, have 
the book “read” to them with a voice-over narration, or mute the sound and read the 
book independently. To navigate through the book, users click pulsing arrows at the 
bottom of the page to “turn the page” forward or backward in the story. When 
navigating across the page with the cursor, an enlarged duck wing signals a hot spot to 
the child who can click on that spot to activate it. 


It is also the intent of this technological enhancement outreach project to involve 
caregivers in the shared eBook reading experience. The Caregiver Guide provides 
suggestions and ideas for adults to maximize a child’s learning while reading the eBook. 
For example, in Totally Terrific Duck, Duck borrows clothes from his friends so he can 
look terrific in a photograph. The Caregiver Guide suggests that adults encourage 
children to talk about a time when a friend helped them out with something. These text- 
to-self connections are a common goal for early literacy. 


Logic Model 


Below we show a logic model of the product’s inputs, activities, outputs, and intended 
short- and long-term goals (see figure 3). 
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Figure 3: WordWorld Logic Model 


WordWorld’s eBook Library 


21 


Methods and Procedures 


Drawing by Volunteer Universal Pre Kindergarten student in Florida 


Below we outline the types of data collected and the methods researchers used to 
collect and analyze data for the evaluation of Technological Enhancements to support 
the outreach efforts of three producers. Given the variation in type of technology 
employed and in project timelines, we treated each producer as a separate entity for 
the purpose of the evaluation, though we addressed common threads and themes 
where possible. 


Before any data collection activities were undertaken, all Institutional Review Board 
(IRB) requirements were complied with, and an exemption from IRB review was granted 
for this project. 


Data Collection 

To initiate the evaluation, researchers talked by phone with producers from each of the 
properties to introduce themselves, discuss the purpose of the evaluation, and identify 
the technological enhancement. Researchers also established a set of expectations for 
participation among producers in the data collection process. 


We collected three types of data: 
¢ Print and online materials 
¢ Interview and site visit data 
¢ Survey data 


Print and online materials 

Researchers collected print and online materials for each property and organized them 
in a secure online system for storing data. We requested documents about the 
development of the property and received access to the project website to download 
and store the properties’ artifacts. Some examples of the categories of documents 
collected were training manuals, teacher guides, descriptive internal documents (such 
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as detailed scope-sequences or focus group research), curricula, and product website 
links. 


Interview and site visit data 

Researchers developed instruments for three levels of interviewees—producers, 
teachers, and parents—and a general observation tool for site visits (see Appendix B for 
all interview and site visit protocols). 


We first created a generic set of interview and observation protocols with a core set of 
questions that would be used for each property. We developed the interview and 
observation protocols using a format and set of core themes that had been developed 
during the large-scale evaluation of Ready to Learn and that built on previously tested 
and refined instruments. We then customized the generic survey and observation 
instruments for each producer’s technological enhancement through multiple rounds of 
review by the evaluation team in consultation with CPB staff. During the customization 
process, we altered the interview protocols to reflect the specifics of each program. Out 
of the Blue interview protocols targeted issues about the recent inclusion of online 
games into the Super WHY! Summer Reading Camp curriculum and daily structure, 
WordWorld protocols targeted shared parent-child experiences with the eBook, and 
Sesame Workshop interview protocols targeted the integration of TEC video and games 
into phoneme lessons with the use of in-classroom interactive whiteboard technology. 


Researchers conducted the following interviews: 


Phone interviews 
¢ The Electric Company School’s Initiative Curriculum: Researchers conducted 
three separate phone interviews with three different TEC producers to 
understand Sesame Workshop’s goals in the development of TEC School’s 
Initiative Curriculum and to gain a clear understanding of the overall 
dissemination and communication plan, and target audience. (May 17, 19, and 
24, 2010) 


¢ Super WHY! Summer Reading Camp Technology Add-On: Researchers 
conducted a group phone interview with three Out of the Blue Enterprise 
members to discuss their role and goals in the development of the Super WHY! 
games. (June 1, 2010) 

¢ eBook Library: Researchers conducted a group phone interview with five 
WordWorld members to discuss their role and goals in the development of the 
eBook. (July 27, 2010) 


Researchers conducted the following site visits: 


Observations and on-site interviews 
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¢ The Electric Company School’s Initiative Curriculum: Researchers conducted a 
site visit including an observation of a teacher training for TEC School’s Initiative 
Curriculum. An SFA reading coach led the training with the support of a Sesame 
Workshop curriculum specialist. This was the only structured product training 
provided by any of the properties. We used a general observational protocol for 
the training and did not conduct any interviews during the site visit. (April 7, 
2010) 


¢ The Electric Company School’s Initiative Curriculum: Researchers conducted a 
site visit to observe the implementation of TEC School’s Initiative Curriculum in 
two first grade classrooms. In addition, researchers met with a group of parents 
and caregivers to discuss their perception of how the materials aligned with and 
enhanced their educational goals and priorities for children in their care. (May 
14, 2010) 


¢ Super WHY! Summer Reading Camp Technology Add-On: Researchers 
conducted a site visit to observe the integration of Super WHY! games into the 
Super WHY! Summer Reading Camp in two pre-kindergarten classrooms. In 
addition, researchers conducted a group interview with two pre-kindergarten 
camp facilitators and conducted an interview with a station director. (July 15-16, 
2010) 


¢ eBook Library: Researchers conducted a site visit to observe four parent-child 
pairs using a prototype version of the eBook at a community center that has 
partnered with a local PBS station. Researchers also interviewed parents on their 
experience and collected their responses to a set of protocol questions, which 
were agreed upon by WordWorld staff members and CPB. Due to time 
constraints, interviews were not fully transcribed or coded. (August 23, 2010) 


Survey Data 

Researchers, in consultation with CPB staff, chose to develop and disseminate surveys 
to two sets of program participants after starting the evaluation. Researchers worked 
with staff members from Out of the Blue and Sesame Workshop to develop online 
teachers surveys for the adults responsible for implementing the programs developed 
by each of these properties. We created separate interview questions and distinct 
surveys for each teacher group. 


We developed survey protocols for two of the three properties as an additional data 
collection activity; these served to reach out to a broader range of participants from 
which to gather data. The information collected also allowed for some triangulation of 
data across data collection points to support general statements about the products and 
their implementation in education settings (see Appendix B for survey protocols). 


24 


Surveys were developed for teachers who facilitated the Super WHY! Summer Reading 
Camp and for teachers who implemented the TEC School’s Initiative Curriculum. 
Because the eBook required a different dissemination and implementation approach, it 
was not feasible to develop a survey for this audience. In addition, the eBook was 
behind in its development schedule and therefore we did not have the opportunity to 
collect additional information from adult users of its products. 


Teachers who delivered TEC School’s Initiative Curriculum received questions about 
their experience delivering lessons with integrated TEC videos and games. Researchers 
worked with SFA reading coaches to distribute the online survey and contact was made 
by email communication with teachers. Twenty-five teachers submitted their responses 
(a 100% response rate), and all received a $20 gift certificate to Amazon.com once they 
completed the survey. The collected data were coded for analysis. 


Super WHY! Summer Reading Camp facilitators also received survey questions about 
their experience with Super WHY! games and their integration with the Super WHY! 
Summer Reading Camp curriculum. Researchers designed the survey tool and relied on 
Super WHY! Summer Reading Camp coordinators to distribute the survey to station 
managers. Station managers or outreach coordinators were then asked to forward a 
link to the online survey to their camp facilitators. Twenty-three camp facilitators 
submitted their responses (a 27% response rate), and all received a $20 gift certificate 
to Amazon.com once they completed the survey. The collected data were coded for 
analysis. 


Data Analysis 

Print and online materials 

Researchers read and catalogued print and online materials. These documents provided 
overall descriptions of each program, additional information to inform survey and 
interview question development, and context for site visits and later data collection and 
analysis. 


Interview and observation data 

Interview data were transcribed and entered into Atlas.ti, a qualitative data analysis 
tool. Researchers developed a preliminary set of themes to use during an initial coding 
of these documents and then modified the themes to respond to additional issues 
found in the data. We developed 14 codes for use across all interviews. These covered 
issues such as the ability of the program to address the needs or skills of the target 
audience; the comments about technology including challenges, previous experience, 
current use, and ease of use; responses about a target population such as teachers, care 
providers, or students; comments about the potential for future use of a product; and 
comments about product development (see Appendix C for Observation Summaries, 
and Appendix E for Interview Summaries). 
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Two researchers developed codes and jointly reviewed coding of interviews periodically 
throughout the coding process. Once coded, frequencies of codes across all interviews, 
across teacher interviews, and across producer interviews were run for further analysis. 


Researchers recorded and summarized the observation data to provide information 
about key elements of the project such as children’s responses to the product or 
challenges observed. Observation data provided researchers with contextual 
information about each product. This allowed the researchers to situate interview, 
textual, and online information within implementation settings and provided 
opportunities to confirm interview responses with observed activities. 


Survey data 

Survey data were collected in SurveyMonkey. Downloaded data were reviewed and 
summaries were prepared of response frequencies to survey items. Each survey also 
contained several open-ended questions and researchers grouped these by common 
themes (see Appendix D for survey summaries and Appendix F for Survey protocols). 


Limitations 

Limitations to the data collected for this evaluation exist, as findings are based on 
qualitative data gathered from a non-representative sample of sites. While the 
information presented is based on the data gathered, it is possible that the sites visited, 
or the teachers interviewed, were not representative of all sites or all teacher 
experiences. In addition, one producer had not yet completed its product, and 
evaluation findings are based on a prototype that may not adequately represent the 
final product. 


Findings 

Findings presented below are described in three sections. First we present findings 
based on each property as it related to the three overarching research questions. (See 
the appendices for additional details and a further elaboration of the findings.) Then we 
present cross-property findings, and finally we present a summary and 
recommendations. 


Super WHY! Summer Reading Camp Technology Add-On 
Data from classroom observations, interviews with camp facilitators and program 
producers, and responses to a camp facilitator survey, all contributed to this section. 


In addition, informal conversations with a station outreach coordinator and a Title | 
coordinator who support program implementation helped to inform our understanding 
of the program and how it has reached young children and the adults who work with 
them. 
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General findings on the technological enhancement 


Overall, responses from camp facilitators were positive with regard to the content, 
design, and implementation of the Super WHY! Summer Reading Camp with technology 
add-on. Camp facilitators indicated in interviews and in the survey that children 
responded with enthusiasm to the computer games, and that many had made 
connections between the Super WHY! characters introduced in the classroom to those 
they had seen at home on television. Facilitators stated that the games were colorful, 
stimulating, and encouraged child engagement in activities, and provided age- 
appropriate practice of early reading skills. These facilitators also stated that the 
experience provided an opportunity for some children to work with a computer for the 
first time (see Appendix A for parent quotes). 


However, some facilitators stated that the program included games that were too 
challenging for their campers. One facilitator indicated that her campers were not yet at 
the point where they could read words: “They are not reading words. They’re just really 
getting a grasp of their sounds.” She stated that some of the games might be more 
appropriate for her campers at the midpoint or end of kindergarten. Another facilitator 
indicated that the spelling requirements were too challenging for her children: “The 
Alpha Pig, that’s the game that they’re most successful at. It’s the letter recognition and 
the sound recognition. That’s pre-K appropriate.” Survey responses indicated that 
children played the Alpha Pig’s Alpha Bricks and Wonder Red’s Freeze Dance Rhyming 
games most often, and used the Story Book Creator least often, suggesting that games 
requiring more basic early literacy skills were more popular choices for participating 
children. 


Do the supplemental materials or activities address a gap or missing link in the 
overall dissemination and communication plan of the product, filling a void that 
would otherwise go unmet by existing products and services? 


Producers indicated that in previous years parents had requested activities that would 
extend children’s experiences from the Super WHY! Summer Reading camp, and would 
allow them to continue engaging in these activities at home: “Some of the feedback that 
we had gotten from parents is how can | do this with my kids at home and what can | do 
to extend the learning when they’re at home.” 


Integrating computer games into the Super WHY! Summer Reading Camp program 
provided children with experience playing the games, modeled through facilitator 
introduction how the games were related to a particular reading skill, and provided a 
classroom-home link supporting families’ awareness of and access to games through the 
PBS website. In this way the program does address a gap (reaching parents and 
connecting home and classroom activities) and also furthers the reach of the program 
by connecting with adults beyond the classroom. Facilitators indicated that some 
parents do have computers available and do access the games at home: “They’ve been 
playing at home. And the parents like, ‘Oh, we let them play while we get ready for 
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dinner.’” However, in survey responses the majority of facilitators indicated they were 
not aware of whether parents or their children were playing with the program’s 
computer games at home. 


Do the supplemental materials meet the needs of the end user (caregiver, 
teacher, or child) in a way that provides a novel/innovative/valued experience? 


Facilitators indicated that they thought students benefited from the Super WHY! 
computer games, and that the games reinforced skills introduced during the camp day; 
however, of those who responded to the survey, only about half of facilitators who had 
participated in the camp last year felt that the computer game additions this year added 
substantially to the experience. Yet nearly all facilitators indicated a preference for 
facilitating the summer camps with the computer games rather than without. 


Most camp facilitator survey respondents stated that the training provided was 
adequate and that they did not require more training; however, a small number of 
facilitators indicated a desire for additional training: “I think a day of training. Maybe 
half a day of introducing the curriculum, what is Super WHY!. Because | didn’t know 
what Super WHY! was until | got the curriculum and the book. | was like, ‘Who is this?” 


Facilitators indicated that the games were engaging, reinforced skills, were paced 
appropriately, and for the most part were academically appropriate: “So the simple fact 
that they were able to actually play, laugh, learn, and have fun was the best part.” 


In survey responses and interview comments, camp facilitators generally believed that 
at least some of the games were a good match for the children, and they indicated that 
the overall camp experience including the computer games, was positive, engaging, and 
supported learning. 


Do the supplemental materials alter the interactions between caregivers and 
young children with respect to early literacy experiences? 


Facilitators did not indicate that their role was altered as a result of the supplement to 
the Super WHY! Summer Reading Camps. However, during observations the facilitator- 
camper interactions that took place were likely different from those that might have 
taken place in the absence of computer games. Facilitators appeared to engage in 
conversation both about the logistics of a computer game (Where should | click first?) 
and also about the elements of early reading skills evident in the game (What rhymes 
with ake? Is it snake?). In addition, observers noted facilitators engaging with campers 
in conversations about technological challenges, with campers attempting to 
troubleshoot problems along with their facilitator. 


Because the Super WHY! Summer Reading Camp curriculum included non-digital games, 


video viewing, and hands-on activities in its earlier iteration, it is unlikely that the 
addition of electronic games would alter camper-facilitator interactions considerably. 
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One element of the computer games is that they are relatively easy to manipulate and 
are intended for self-directed use by young children. It is possible that an unintended 
outcome of the addition of computer games is that it provided some children with 
opportunities to interact with a computer without adult support. Most facilitators 
responding to the survey indicated that campers had opportunities to play the 
computer games either independently or in pairs. 


Recommendations 

Two key recommendations emerged from these findings and relate specifically to camp 
facilitator concerns. The first is facilitator’s interest in additional training or an 
introduction to the materials before implementation, and the second is support for 
technical difficulties encountered by facilitators in classrooms. 


Providing on-site training to facilitators for a nationally implemented program can be 
challenging on many levels and may not be the best use for limited resources. However, 
providing facilitators with more guidance on program implementation could help them 
sustain implementation over multiple years, spread information to peers about the 
program, and have greater success with campers during implementation. We suggest an 
alternative to in-person trainings might be an online walk-through of materials that 
facilitators can view on their own time either as a group at a school site, or individually 
at home or in their classroom. This program tutorial can introduce the overall concept of 
the Super WHY! Summer Reading Camp, the basic elements of the program, and any 
materials or other resources needed before implementation. Making a tutorial available 
online would allow facilitators access to program information, and flexibility in retrieving 
that information at convenient times and locations. 


Providing technical support to multiple sites that are geographically dispersed is another 
challenging task and is one that may not be easily addressed. However, it is possible that 
providing DVDs to sites may overcome some of the challenges facilitators face. The cost 
of distributing DVDs to every site may be prohibitive and may not be an economical 
choice, but making these available to locations that anticipate difficulties loading or 
opening programs from the Web may give facilitators a sense of having options and may 
help them overcome some fears about dependence on the Web during an instructional 
unit. 


The Electric Company School’s Initiative Curriculum 


General findings about the technological enhancement 


Overall teachers were very satisfied with TEC School’s Initiative Curriculum, and they 
reported that it engaged students and encouraged participation in activities that might 
otherwise be repetitive and uninteresting. Teachers indicated that the program was fun 
and that students enjoyed their experience when the games and videos were 
introduced. Two thirds of teachers stated that the pacing of the games was appropriate, 
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and most teachers indicated that the vocabulary, number, and length of video segments 
and games were also appropriate (see Appendix A for quotes). However, several 
teachers reported that the videos were more relevant to the phonics lessons than were 
the games. Nearly all teachers indicated their students benefited from TEC School’s 
Initiative Curriculum because of increased engagement and also improved lesson 
structure. Two teachers stated that students’ test scores improved as a result, and 
another teacher indicated this provided a needed review for struggling readers. Another 
teacher noted that the program provided access to the SFA material in a much more 
visual way. It is possible that this approach allows students with different strengths to 
have easier access to the content. 


Despite noting that their students were engaged and enjoyed the program, nearly half 
of teachers reported that some TEC School’s Initiative Curriculum lessons were not 
compatible with their students’ grade level and were also inconsistent with the SFA 
phonics lessons. And approximately one third of teachers indicated that the pacing of 
the lessons could be better adjusted to meet children’s needs. A quarter of teachers 
stated that there were many technical challenges, particularly with graphics and sound, 
and in some instances the Smartboard program did not function properly. 


Do the supplemental materials or activities address a gap or missing link in the 
overall dissemination and communication plan of the product, filling a void that 
would otherwise go unmet by existing products and services? 


In interviews, producers and staff talked about listening to teachers to inform the 
development process and stated that they had attempted to respond to the needs of 
the classroom. During conversations with producers, teachers had indicated a need to 
improve instruction in vocabulary because they saw that the SFA program did not 
provide enough support in this area. In addition, some teachers had indicated the 
repetition of the SFA program could be boring to students and having something to 
encourage repetition in a fun, engaging way would address this need. 


Teachers had also stated that the TEC videos were very useful in supporting phonics 
instruction and that students enjoyed them, found them engaging, and had noted their 
absence when they were not available for additional SFA instructional units. 


Do the supplemental materials meet the needs of the end user (caregiver, 
teacher, or child) in a way that provides a novel/innovative/valued experience? 


Teachers indicated in interviews and survey responses that TEC School’s Initiative 
Curriculum was useful both to their own teaching and to their students’ learning. One 
teacher explained that the games and videos were great for reviewing sounds and 
introducing new sounds, saying that the students “get it” and they know what’s coming. 
Another teacher stated that what is done with the interactive whiteboard would be too 
complicated to reproduce in a printed text. For example, a third teacher noted that 
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having the student “see” how a word changes when you substitute a letter is so 
important. The Smartboard allowed for this. 


Parents also indicated that participation in this program supported their children’s 
home computer use for educational activities, stating that their children would go home 
and turn on the computer to seek out the games they had played at school. 


Teachers valued the experience because of the opportunity it provided to integrate 
their Smartboards with their instruction. And one teacher indicated that it helped 
engage her in the lessons as well as engaging her students. 


Do the supplemental materials alter the interactions between caregivers and 
young children with respect to early literacy experiences? 


Teachers regularly indicated that using the whiteboard and curriculum materials was 
“fun” and “engaging,” and that it made the instruction of phonics less boring. It is 
possible that the less boring nature of these lessons altered the interactions between 
teachers and their students, but we do not have any evidence of this aside from 
teachers’ assertions that the current experience is fun and that nearly all preferred the 
current program with the whiteboard integration to the “business as usual” program 
from SFA. 


It is possible that the program has changed the interactions between parents and 
children because children were reported to be playing the TEC computer games at 
home. One parent noted that if their child could not say a word, then the parent would 
recommend sounding it out and the child would know how to do this as a result of the 
program. Access to The Electric Company on television in homes may encourage 
students to tell their parents about use of the program in school, and this may lead to 
greater communication and understanding of school activities in the home. 


Recommendations 


Responses from teachers in interviews, surveys, and during site visits suggest two 
potential areas of improvement for TEC School’s Initiative Curriculum. First, several 
teachers had concerns about the pacing of the games, and some mentioned that time in 
general was an issue because the overall curriculum requires 45 minutes to implement. 
Teachers suggested that the pacing of some games could be sped up, specifically the 
pacing of the celebration window, which they stated took too long to finish. They 
suggest keeping this element but making it shorter. 


Teachers also suggested that greater interaction with the games would benefit their 
students. Because they see the primary advantage of this program as engaging students 
in tasks that they might otherwise consider dull, providing more ways to engage them 
will build on this benefit. Somewhat relevant to this is a comment several students 
made during our observation. Students noted that regardless of how one performed on 
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a game, the score was always the same. Many students are very familiar with computer 
game play and quickly recognized that this was not a typical outcome for a computer 
game; it is possible that this lack of reflection of a student’s performance may impact 
that student’s engagement in the game. Modifying games to provide greater interaction 
and an accurate reflection of that interaction may support the goal of engaging students 
fully in lessons and in practicing target skills. 


eBook Library 


General findings about the technological enhancement 


Parents and young children positively received the eBook developed by WordWorld. 
Parents stated they liked the book and felt that the pacing of reading, the visual images 
and animations, and the reading out loud of text were all positive qualities. Children also 
appeared to enjoy the book, and all but the youngest (age 3) showed a sustained 
interest in reading or looking at the book. 


Spanish-speaking parents were concerned they would not be able to follow the content 
of the book because both the text and suggestions for parents or caregivers were 
provided only in English. These adults indicated that they did not want to read the book 
to their children in Spanish; rather, they wanted to encourage their child’s reading in 
English by having a better understanding of the content and by helping them 
understand the meaning of the words in the story. 


Do the supplemental materials or activities address a gap or missing link in the 
overall dissemination and communication plan of the product, filling a void that 
would otherwise go unmet by existing products and services? 


WordWorld’s website includes eBooks that are available for children or adults to click 
through. The newly developed eBook includes additional clickable elements and hot 
spots but does not differ significantly from the previous eBooks. However, the approach 
to disseminating the eBooks will be augmented through the provision of the eBook 
Library on CDs and increased outreach efforts resulting from partnering with new 
organizations. These partnerships will increase visibility of the WordWorld website and 
eBook Library, and they will increase distribution of information about WordWorld’s 
website and new CDs. 


Do the supplemental materials meet the needs of the end user (caregiver, 
teacher, or child) in a way that provides a novel/innovative/valued experience? 
The eBook Library provides children and their parents with an engaging activity. Parents 
indicated that having the book read aloud and allowing for children to click on individual 
words to hear them read, was valuable. Two of the four parents interviewed stated that 
they had computers at home, and this suggests that some portion of the target 
audience for the eBook will have home access; however, it is likely that other children 
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and families will have access to the WordWorld website, or will be able to view a CD, at 
an alternate location such as a library or community center. 


Do the supplemental materials alter the interactions between caregivers and 
young children with respect to early literacy experiences? 


The eBook can alter the interactions between a parent or care provider and child. The 
reading out-loud function can support a shared reading experience, the animations and 
hot spots that produce humorous elements in the story can engage a child and his or 
her parent, and an adult can make use of one or more suggestions in the Caregiver 
Guide to help structure interactions with a child. In these ways, the product seeks to 
support adult-child interactions, and during the observation, some of these interactions 
took place with children acting as the driver behind communication with their parent. 


Recommendations 


Although the eBook was well received, we offer some suggestions drawn from parent 
comments, and from our experience as evaluators of other technology initiatives that 
may help the product achieve its outreach goals more effectively. 


Because a goal for the eBook Library is to support early literacy learning that includes 
concepts of print, the eBook design can further this goal by providing common elements 
of a book cover or title page, including the book title, author, and illustrator. Children 
are often assessed on these elements, and giving them opportunities to be exposed to 
this information will help them prepare for such assessments. 


A second WordWorld goal is supporting parent-child interactions with text. Providing a 
simplified Caregiver Guide that is paired down and offers suggestions for co-reading 
with a child and that can be implemented individually or as a group will help invite 
parents into the activity. 


Third, we suggest addressing the needs of non-English-speaking parents by providing a 
Spanish version of the Caregiver Guide so that this population of parents can also help 
children read and understand an eBook. 


Our fourth suggestion is that the eBook include a single-page Spanish translation of the 
text so that parents can support their children’s comprehension of the information in 


the eBook, which will also help to achieve WordWorld’s goal of supporting children’s 
early literacy experiences. 


Discussion and Recommendations 
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Discussion 

Despite the substantial differences in the goals and methods of the three supplemental 
technological enhancement efforts, some elements are common to all three. The most 
prominent of these is the focus producers had on supporting literacy skills for 
underserved children and the adults who care for them, and the use of technology to 
reach out to these populations in innovative ways. Although the producers used unique 
technologies to address different child and adult populations (from age 3 through 8; and 
teachers or parents), all sought to ensure that every child had access to experiences that 
would support early literacy learning. All three also took advantage of current and 
innovative technology to support this goal. If we return to the three central evaluation 
questions when considering the outcome of all three properties, we must consider 
some other common findings. Although our analysis of data from each property did not 
produce positive outcomes in response to each of the three guiding research questions, 
our findings indicate that all three properties did produce positive outcomes in response 
to at least one of these key research questions. 


Question one asked whether the supplemental materials or activities address a gap or 
missing link in dissemination and communication that would otherwise go unmet by 
existing products and services. Of the three properties, two appeared to meet this need: 
the Super WHY! Summer Reading Camp with technology add-on and TEC School’s 
Initiative Curriculum both address specific gaps raised by teachers and identified by 
producers as they sought to develop their technological enhancements. There was no 
evidence of a gap or need filled by the development of an eBook Library CD; however, it 
should be noted that parents did say they thought having an eBook on a CD would 
encourage a child to use it more often because the simple act of seeing the CD would be 
a reminder. 


Question two asked whether the supplemental materials meet the needs of the end 


user in a way that provides a novel experience. All three properties were able to address 
this issue in ways that reflect their individual goals. The Super WHY! Summer Reading 
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Camp games provided young children with engaging experiences that supported 
practice of early reading skills. Although not all games were considered academically 
appropriate for all children, camp facilitators were able to support their children’s use of 
at least two of the four games and indicated that children benefited from the 
experience and would not have had the same experience without the games. TEC 
School’s Initiative Curriculum exposed students to phoneme practice in ways that were 
engaging, visual, and fun, and provided some relief from what was perceived as an 
otherwise less visually interesting curriculum. Lastly, the eBook provided children and 
their caregivers with a reading experience that could be modified to meet their needs 
(with or without reading aloud) and encouraged parent engagement in the early reading 
experiences of young children. 


Question three asked whether the supplemental materials alter the interactions 
between caregivers and young children with respect to early literacy experiences. In this 
case we suggest that only the eBook provided some evidence of altering interactions 
between a child and caregiver because of the engaging animations, story line, and visual 
and auditory information about letters, sounds, and the words they make. However, we 
also suggest that the other properties have the potential to alter informal interactions 
between adults and students because of their engaging nature and their use of humor, 
physical activity, and singing to encourage participation. Both TEC/SFA video segments 
and the games included in the Super WHY! Summer Reading Camp seek to engage 
children in age-appropriate activities such as singing, or game-play in ways that 
reinforce a specific concept (such as the alphabet or a phonic element). Once children 
are engaged they are more focused on a task, more likely to retain the information that 
is presented and are more likely to receive positive reinforcement from a teacher (who 
may not provide as much positive reinforcement in a setting where children are not 
engaged or focused). In this way the technological enhancements may support altered 
interactions between adults and children in different settings. 


Exploring innovative uses of technology to achieve the goals laid out by the three 
properties included in this study is a highly challenging undertaking, but it also can be a 
rewarding process. Technology has provided a broad range of new approaches to 
reaching out to children, parents, and educators, but exploring which of these 
approaches is best suited to a particular audience or task will require a considerable 
amount of experimentation. Efforts such as the Technological Enhancements project will 
provide guidance and leadership for future work in this area. 


Recommendations 

Ongoing support for the development of programs and outreach efforts that include 
digital tools requires a willingness to experiment with new and existing technologies, 
and with innovative approaches to integrating media into learning experiences for 
young children and their care-providers. Undertaking such a development process may 
be out of the ordinary for many traditional public media producers. Yet fostering an 
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environment where this kind of development is encouraged and supported must take 
place if new ways of reaching new and existing audiences are to be discovered. We 
recommend that CPB and its partners continue supporting this kind of innovation and 
exploration by integrating the following into their current and future programs: 


¢ Support communication across producers and public media stations 
experimenting with digital tools in a range of learning contexts. In this way 
different properties may be able to build on the experiences of others, which 
may lead to opportunities for collaboration across properties and will contribute 
to a shared culture of innovation and iterative development. 


* Conduct ongoing needs assessments of target audiences. Although technology 
may offer innovative ways to reach out to those whom public media is seeking to 
support (e.g., families, teachers, and learners), access to different kinds of 
technology is still limited for some populations. Ongoing needs assessment can 
help producers and stations determine which audiences have access to which 
technologies, and which audiences are benefiting from newly introduced 
technological enhancements. 


¢ Recognize that developing new and innovative uses of technology requires room 
for experimentation and for mistakes. Providing opportunities for producers to 
experiment and to recognize and learn from mistakes may help to create an 
environment where innovation leads to new ways of reaching out to children 
who are most in need of support. 
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Appendix A: 


Document Prepared in Response to CPB Request Before GPRA Presentation 


Snapshot of Teacher and Parent responses to the Technological Enhancements 
integrated into Sesame Workshop’s The Electric Company School’s Initiative 
Curriculum, and Out of the Blue’s Super WHY! Summer Reading Camps. 


Sesame Workshop’s The Electric Company School’s Initiative Curriculum: 

Sesame Workshop partnered with Success for All (SFA) to develop a hands-on 
curriculum using interactive whiteboard technology. This has resulted in a set of lessons 
that allow teachers to deliver SFA phoneme instruction using videos and games that 
borrow characters from The Electric Company (TEC) story line. The Electric Company 
School’s Initiative Curriculum uses SFA’s Fast Track Phonics lessons infused with TEC 
content, and allows teachers to integrate media and games as they build students’ skills 
in letter-sound correspondence (phonemic awareness) and word-level reading and 
spelling. The target setting for The Electric Company School’s Initiative Curriculum is a 
classroom equipped with interactive whiteboard technology where students are being 
taught phoneme concepts (Grade 1 and 2). 


Out of the Blue’s Super WHY! Summer Reading Camps 

Out of the Blue Enterprises developed an Online Games Extension that built on their 
existing Super WHY! Summer Reading Camp. The camps target children age 4 and 5 who 
attend a weeklong session during the summer. The camps provide access to multiple 
literacy-oriented games, hands on activities, video viewing that is linked to early literacy 
skill activities, and computer games that reinforce early literacy skills. Camp 
facilitators/teachers are provided with detailed curriculum materials that lay out each 
aspect of the camp day, the materials needed to complete activities, and the language 
that teachers can use to introduce the video segments, computer games and hands on 
activities to their campers. Target settings for the Super WHY! Summer Reading Camps 
are pre-Kindergarten classrooms where a camp facilitator has access to a computer and 
video monitor. 


Data Collection 
Researchers collected three types of data during the evaluation process: 


¢ Print and online materials: Collected from each property and organized ina 
secure online system for storing data. Researchers requested documents relating 
to the development of their property, and received access to the project website 
to download and store the properties’ artifacts. 
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¢ Interview and observation data: Developed for three levels of interviewee: 
producers, teachers and caregivers. A general observation tool was developed 
for observing both classroom activities and teacher training events. 


¢ Survey Data: Researchers, in consultation with CPB staff, chose to develop and 
disseminate surveys to two sets of program participants after evaluation 
activities had been initiated. Researchers worked with members from Out of the 
Blue Enterprises and Sesame Workshop to develop online teachers surveys for 
the adults responsible for implementing the programs developed by each of 
these properties. Separate interview questions and distinct surveys were created 
for each teacher group. 


Data Collected 

The following are selected quotes from teachers gathered through the online surveys 
and in person interviews, along with parent/caregivers’ responses to interview 
questions. 


The Electric Company School’s Initiative Curriculum: 


Teachers 

| completely love it because it has saved us tons of time, engaged the students ina 
positive manner, and improved our reading scores overall. The students love to do 
phonics and pay attention because they know they are going to play a game. They also 
watch the videos carefully and associate with the sound that they have learned. It has 
made learning to read a lot more fun for the students. — The Electric Company/Success 
for All Teacher 


| LOVED the videos, the music, and the games. | really enjoyed seeing a lot of the "quick 
erase" component into the program. Having the students "see" a word changing by 
substituting a letter/s is crucial in their spelling and reading. | also liked how during most 
of the videos there was a quick definition to the word. It is something | incorporate in my 
own teaching. It saved me time. — The Electric Company/Success for All Teacher 


The students responded very positively to the videos. The students also enjoyed the 
games. It's funny to hear some of the students memorize the lines of some of the 
characters in the videos. | just recently finished all the Electric Company videos and | 
have been teaching the phonics without the Electric Company segments. The kids are so 
disappointed that there aren't any more electric company videos/games in our lesson 7 
phonics. | have been playing the DVD that was given to us just to review certain sounds. 
The kids really love them and they are so into it that it makes teaching phonics a lot of 
fun. — The Electric Company/Success for All Teacher 
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Born in 1965, | probably learned to read by watching [Electric Company] EC (I was 
surprised that | can remember all of the lyrics to the original 70s show), sol ama bit 
sentimental about it. Also, as a parent of a first grader hooked on Disney pseudo sit- 
coms, | encouraged her to switch to Public TV for EC. She loves it! — The Electric 
Company/Success for All Teacher 


| liked the fact that now they are more interactive and | get to use a Smart Board for the 
benefit of my students' learning. | can't wait to use it at the beginning of the year with 
my new class. — The Electric Company/Success for All Teacher 


Parent/Caregiver 

It just teaches them a lot. It teach them a lot and I'll be right there with her as she is 
watching the show, I’Il be right there with her. Looks like some of the stuff | had. — The 
Electric Company/Success for All Parent/Caregiver 


| felt fine considering an educational tool [would be] on [in school], because they watch it 
on TV [at home] and then they come to school and they see it in there and then they get 
a feel of what’s going on the TV and they learn more. They’re able to learn more at the 
school. — The Electric Company/Success for All Parent/Caregiver 


Super WHY! Summer Reading Camp Games: 


Teachers 
The children love to use the computer and | was happy that it reinforced the goals of the 
day. — Super WHY! Summer Reading Camp Teacher 


The kids LOVED the chance to play on the computers. They took turns with partners and 
were eager to participate. The skill -level-appropriate games reinforced the day's lesson. 
— Super WHY! Summer Reading Camp Teacher 


Definitely, they were more attentive to the show when told what game would be played 
that day. So very close attention was paid to those portions [where] they felt they were 

in the action! Having taught this camp for the last several years, | noticed better results 

on the post tests. - Super WHY! Summer Reading Camp Teacher 


It gave another dimension to the show and helped the children experience more success 
and confidence in the skills they were learning. - Super WHY! Summer Reading Camp 
Teacher 


| loved how the games extended the skill of the day for each character. | loved the 
stimuli. They were colorful, there was zipping, and popping. The kids loved the way they 
were able to interact with each character and sing their special songs especially with 
Wonder Red (all-ball- call...... so catchy!) — Super WHY! Summer Reading Camp Teacher 
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Well, it’s repetition, repetition, repetition. And they’re able...every time they do it, it 
builds on that and it reinforces what we’re learning during the day. And especially with 
[special needs] students you need repetition, repetition, repetition. And it’s predictable. 
Once they see Alpha Pig they know how the game is played and they can kind of predict 
and they know. And that gives them the confidence to keep trying versus if it’s a work 
book, they’re not really interested in that. And they like the graphics. “Oh, you’re flying.” 
And the sounds and so forth. — Super WHY! Summer Reading Camp Teacher 


And my class is a blended class. | have nine students, four of them are ESE and one of 
them is autistic. And all four of my ESE [students] are either speech or language delayed 
and developmentally delayed. They enjoy the computer games because they can kind of 
pick what they want to do. They can put the headphones on and they’re in their own 
world, they can work at their own level. | like the beginner, medium, hard, because they 
can pick their own. — Super WHY! Summer Reading Camp Teacher 
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Appendix B: Data Collection Protocol 


RTL Technological Enhancements Observation Protocol 


Date: Observer(s): 
RTL Producer: Name of Event: 
— WordWorld 

Sesame Workshop 


OQ Out of the Blue Enterprises, Inc. 


Location: Purpose: 
About the Participants 
Total # of Participants: Observer: 


# of parents: 


# of educators (indicate type, e.g., 
preschool teachers): 


# of children: # of others 

(Indicate age range): (describe): 

Ethnicity: Gender: 

About the RTL Producer 

Event Coordinator: Length of event: 
Community/School partner: Materials distributed: 
Language(s) of content delivery: Artifacts collected: 


Description (or diagram) of Setting 


Brief Description of Event/Activity 
A. Format/type of event/activity: 


a. 
b. 
C. 


Target Audience 
Purpose of event/activity 
Instructional strategies 


B. Technological Enhancement (Product): 


a. 
b. 
C. 


Description of Product 

Description of End User 

Medium /Platform of Technology (Computer, handheld, mobile, 
etc.) 

Observed interactions Between Caregivers and Young Children 
(with respect to early literacy experiences & the product) 
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e. Literacy Topics or Skills Addressed Using the Product 
f. User Access to the Product (Public/Private) 

C. Materials in use (Teacher/Student/Family-directed): 

D. Materials available to send home: 


Detailed Description and Reflection of Event 

Provide a “running record” of the event (unrestricted length). As much as possible, use 
active voice and make the agents of action clear. Quote participants directly, as 
feasible. Describe not only the formal activities going on, but also how the audience 
responds and interacts with the producer staff. Use the reflective notes column for any 
impressions or interpretations you have of what you have recorded in the descriptive 
column. "Time stamp” each new episode of activity in a new row and be sure to align 
your reflective notes with the description. Pay attention to particular behaviors, words, 
or symbols that have traction in an activity. 


Time Descriptive Notes Reflective Notes 
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Super WHY! Summer Reading Camp Producer Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Producer Interview Protocol 
This protocol is appropriate to use with producers involved in coordinating the 
SuperWHY! Reading Camp games. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about your role in producing the Super WHY! Reading Camp 
and the computer games that have been added to camp activities. As part of the 
evaluation of the Ready to Learn initiative’s Supplemental Outreach effort, we are 
conducting interviews with individuals involved in producing the computer games 
that have been added to the Super WHY! Reading Camps. This interview will take 
about one 1 hour. 

Everything you tell us today will be confidential. To protect your privacy, we 
won't connect your name with anything that you say. At any time during our 
conversation, please feel free to interrupt me if you have any questions or if you 
would rather not answer any specific question. You can also stop the interview at 
any time for any reason. 

Please remember we want to know your opinion about the questions we are 
asking and that there are no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we’ve heard, as we write up our reports. 


Background Information on Ready to Learn Technological Outreach 
1. Could you describe the games developed for the Super WHY! Reading Camp? 


2. What is the main message you hope to convey with the games for the Super 
WHY! Reading Camp? (Probe for main learning goals.) 


3. What was your role in producing the games for the Super WHY! Reading 
Camp? 
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4. What is your content focus for the games? 


5. How do the games for the Super WHY! Reading Camp differ from what you 
have done in past years? What component of the Camp is new and what is 
part of the original outreach effort? 


6. Could you tell me about how the games for the Super WHY! Reading Camp 
came to be part of your supplemental outreach activities? How do they 
extend the overall dissemination and communication plan for RTL? 


Games Target Audience 


1. For what target audience or audiences did you develop the games for the 
Super WHY! Reading Camp? Probe for whether target audiences include 
children (including their ages), parents, educators, and/or hard to reach 
populations. 


2. How do you envision the new games being implemented by the SuperWHY! 
Reading Camp instructors? 


3. What need do you see the games for the Super WHY! Reading Camp fulfilling? 


4. What materials are distributed to the camp facilitators to support their use of 
the Super WHY! games?? 


a. How do you know if camp facilitators have used the games for the 
Super WHY! Reading Camp? Do you provide training to facilitators? If 
yes, what was the goal of the training? 


b. How do you know if campers (children) have used the games for the 
Super WHY! Reading Camp? Do you provide any instructions for use? 
If yes, what was the goal of that instruction? 


5. Do adults or kids enjoy using the games for the Super WHY! Reading Camp? 


6. In your view, how successful are the games for the Super WHY! Reading 
Camp in reaching parents, educators and children? 


a. How do you know? 


Responses to the Outreach 


7. Did you have a role in implementing the games for the Super WHY! Reading 
Camp with adult end users? If so, what responses do they have to the camp 
program and materials? 


8. Describe any formal or informal ways you have of finding out how well Super 
WHY! Reading Camps are working once they’re launched. 
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9. Do you feel Super WHY! Reading Camp games accomplished their goals? 
What did they do well? What would you change if you could? 

10. What feedback about the Super WHY! Reading Camp games have you 
received from parents or caregivers? 

11. Is there anything else you'd like to tell us? (Prompt: about the games, 
activities, outreach efforts or target audience) 


Thank you for your time. 
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Super WHY! Summer Reading Camp Teacher Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Teacher Interview Protocol 
This protocol is appropriate to use with teachers/facilitators for Super WHY! Reading 
Camps. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about the Super WHY! Reading Camp. I work for the 
Education Development Center and we are evaluating the use of technology as part 
of camp activities. We are conducting this interview with you so that we can learn 
more about your involvement and role with the Super WHY! Reading Camp. This 
interview will take about one 1 hour. 

Everything you tell us will be confidential. To protect your privacy, we won't 
connect your name with anything that you say. At any time during our conversation, 
please feel free to interrupt me if you have any questions or if you would rather not 
answer any specific question. You can also stop the interview at any time for any 
reason. 

Please remember we want to know what you think and feel and that there are 
no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we’ve heard, as we write up our reports. 


Implementation Background 
1. How did you find out about the Super WHY! Reading Camp? Did you facilitate a 
Super WHY! camp last year? 


2. Have you used computer games in the past? How does what you’re doing now 
compare with those prior games? 


3. WHY! did you decide to facilitate a Super WHY! Reading Camp? 


4. Can you tell us about the campers who participated in your Super WHY! Reading 
Camp? Probe for age and backgrounds of children and parents served. 
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5. What are some of the primary educational needs of the young children and 
families who attended the Super WHY! Reading Camp? 


Using the Program 
6. What was your role in implementing the Super WHY! Reading Camp and the new 
games? 


7. How easy was it to use the Super WHY! Games? 


8. Did you receive training in use of the Super WHY! games? Was it helpful, and if 
so, in what way? Would any additional training or support have been helpful? 


9. What is the content focus of the Super WHY! Games? Was this focus helpful to 
your campers? WHY! or WHY! not? 


10. Are any Super WHY! Reading Camp materials distributed to parents for use at 
home? What was the purpose of these materials? 


11. What media or technology (computers, consoles, etc.) did you use to implement 
the games and how many did you have access to? Had you used them before for 
teaching? How do you feel they worked for you and your campers? 


12. Describe any challenges you faced in adding the Super WHY! games to the 
reading camp. Do you have any recommendations for avoiding or addressing 
these challenges in the future? 


Reflecting on the Program 
13. Overall, how have campers and parents responded to the Super WHY! games? 
Did use in different contexts (for example, in classrooms, or at home) prompt 
different responses to the Super WHY! games? 


14. Do you believe students benefitted from using the Super WHY! games? In what 
ways, if any? How can you tell? 


15. Have the Super WHY! games benefitted your camper group management, 
instruction, or delivery of curriculum in any way? Did it meet your needs? If so, 
how? If not, WHY! not? 


16. Based on your experience with including the games in the Super WHY! reading 
camp, would you do it again? Do you think it would be useful for you or your 
campers to do it again? WHY! or WHY! not? 


17. How did you like using the Super WHY! games? How did the campers like using 
them? 


18. Did you find the program to be unique? Valuable? Innovative? If so, how? 


19. What feedback did you receive about the Super WHY! games from campers 
and/or parents? Any memorable stories? 


20. Had the campers used computer games for learning purposes before? 
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21. How well did the Super WHY! games fit in with your curriculum? Did the 
program meet any of your campers’ needs? What needs were met, how were 
they met, or WHY! weren’t they met? 


22. If you used Super WHY! materials last year, what do you think about the added 
games this year? What are the benefits of the Super WHY! games? What are the 
weaknesses? 


23. Is there anything else you’d like to tell us about your experience using the Super 
WHY! Reading Camp and computer games? 


Thank you for your time. 
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Super WHY! Summer Reading Camp Parent Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Parent Interview Protocol 


This protocol is appropriate to use with parents whose children have participated in 
ae Super WHY! Reading Camp. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about the Super WHY! Reading Camp. I work for the 
Education Development Center as an evaluator for this project. As part of the 
evaluation, we are interviewing the parents of children who have participated in a 
Super WHY! Reading Camp. We would like to hear what you think about the camp. 
This interview will take about 30 minutes. 

Everything you tell us today will be confidential and we will not connect your 
name with anything you say. At any time, please feel free to ask me any questions. 
You can also stop the interview at any time for any reason. 

Please remember we want to know what you think and feel and that there are 
no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we've heard, as we write up our reports. 


Exposure to Outreach Efforts: Technological Enhancements 


1) Is this your child’s first time participating in the Super WHY! Reading Camp? 
Have you participated in other activities like this one? (If so, how many?) 


2) How did you hear about the Super WHY! Reading Camp? 


op 


3) WHY! did you decide to have your child attend the Super WHY! Reading 
Camp? Is there anything that caught your attention? 


4) What does your child think of the camp (e.g.: fun, boring, wants to go home, 
wants to do it again, etc)? Do you think he/she liked it? Do you think he or 
she learned anything, and if so, what? 


5) Did you see or hear of any activities that have to do with learning to read 
while you were here? If so, what did you see/hear? Which activities? 


6) Did you see or hear anything that surprised you today? 


7) Did you see any activities that you or your child might continue to do at 
home? If so, which ones? 


8) Do you think your child has learned anything from the Super WHY! Reading 
Camp games? 


9) Ifthis is not your first time participating in the Super WHY! Reading Camp: 
a. WHY! did you decide to return? 
b. How is it different from you last time participating? 


10) Were you familiar with the games for the Super WHY! Reading Camp used 
today? Would you use them again? Did they seem easy to use? WHY! or 
WHY! not? 


11) Where do you look to find out more about programs and activities for young 
children? 


12) Is there anything else you would like to tell us about the Super WHY! 
Reading Camps and the games? 


Thank you for talking the time to talk with me about this camp. 
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The Electric Company School’s Initiative Curriculum Producer Interview 
Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Producer Interview Protocol 
This protocol is appropriate to use with producers involved in coordinating The 
Electric Company/Success for All Curriculum, videos and Digital Literacy Resource. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about technological outreach materials/activities that are 
part of Ready to Learn. | work for EDC and we are the evaluators for the Ready to 
Learn Supplemental Outreach. As part of the evaluation, we are conducting 
interviews with members involved in creating and providing training for the RTL 
producer’s materials/activities. The purpose of this interview is to learn specifics 
about your involvement and role with The Electric Company/Success for All 
Curriculum, videos and Digital Literacy Resource and educational efforts that are 
related to Ready to Learn. This interview will take about one 1 hour. 

Everything you tell us today will be confidential. To protect your privacy, we 
won't connect your name with anything that you say. At any time during our 
conversation, please feel free to interrupt me if you have any questions or if you 
would rather not answer any specific question. You can also stop the interview at 
any time for any reason. 

Please remember we want to know your opinion about the questions we are 
asking and that there are no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we've heard, as we write up our reports. 


Background Information on Ready to Learn Technological Outreach 


7. Could you describe The Electric Company/Success for All Curriculum, videos 
and Digital Literacy Resource (also called DLR) for me? 


8. What was your role in producing the The Electric Company/Success for All 
Curriculum, videos and DLR? 
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9. What is the main message you hoped to convey with The Electric 
Company/Success for All Curriculum, videos and DLR (Probe for main 
learning goals.) 


10. What is your content focus for The Electric Company/Success for All 
Curriculum, videos and DLR ? 


11. How does The Electric Company/Success for All Curriculum, videos and DLR 
differ from what you did in past years with The Electric Company? How do 
the materials/content differ from last year’s outreach effort? 


12. Could you tell me about how The Electric Company/Success for All 
Curriculum, videos and DLR came to be part of your supplemental outreach 
activities? How do they extend the overall dissemination and communication 
plan for RTL? 


Audience for Technological Outreach 


13. When developing The Electric Company/Success for All Curriculum, videos 
and DLR, who was your primary target audience (eg: teachers or students)? 
Probe: WHY! did you make this decision? 


14. How did you originally envision the new applications being used? 


15. What need do you see The Electric Company/Success for All Curriculum, 
videos and DLR fulfilling? 


16. Describe what you provide for end users of The Electric Company/Success 
for All Curriculum, videos and DLR. 


a. What materials are distributed to the audience? 


b. Do you know whether teachers had any prior experience with the 
media or technology (Interactive WhiteBoards, computers, projectors, 
etc.) ? 


c. Did you provide training? If yes, what was the goal of the training? 


17. Did adults or kids enjoy their experience with The Electric Company/Success 
for All Curriculum? 


18. How successful is The Electric Company/Success for All Curriculum, videos 
and DLR, in your view, in reaching your target audience? 


Responses to the Outreach 
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19. Do you have a role in implementing your outreach materials with adult end 
users? If so, what responses do they have to the outreach program and 
materials? 


20. Describe any formal or informal ways you have of finding out how well The 
Electric Company/Success for All Curriculum, videos and DLR was worked 
once it was launched or distributed. 


21. Do you feel the materials accomplished their goals? What did they do well? 
What would you change if you could? 


22. What feedback about The Electric Company/Success for All Curriculum, 
videos and DLR have you received from parents or caregivers? 


23. Is there anything else you'd like to tell us? (Prompt: about the materials, 
activities, outreach efforts or target audience?) 


Thank you for your time. 
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The Electric Company School’s Initiative Curriculum Teacher Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Teacher Interview Protocol 
This protocol is appropriate to use with teachers involved in implementing The 
Electric Company/Success for All Curriculum. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about The Electric Company/Success for All Curriculum. I 
work for the Education Development Center and we are evaluating the use of 
technology as part of this curriculum. We are conducting this interview with you so 
that we can learn more about your involvement and role with The Electric 
Company/Success for All Curriculum, videos and Digital Literacy Resource. This 
interview will take about one 1 hour. 

Everything you tell us will be confidential. To protect your privacy, we won't 
connect your name with anything that you say. At any time during our conversation, 
please feel free to interrupt me if you have any questions or if you would rather not 
answer any specific question. You can also stop the interview at any time for any 
reason. 

Please remember we want to know what you think and feel and that there are 
no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we’ve heard, as we write up our reports. 


Implementation Background 
24. How did you find out about The Electric Company/Success for All Curriculum, 
videos and Digital Literacy Resource (also called the DLR)? 


25. Have you used digital media materials in the past? How does what you’re doing 
now compare with those prior materials? 


26. WHY! did you decide to participate in the program? What did you think would be 
valuable about participating? 
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27. 


28. 


Can you tell us about the students who participated in your delivery of The 
Electric Company/Success for All Curriculum? Probe for age and backgrounds 
of children, teachers, and/or parents served. 


What are some of the primary educational needs of the young children and 
families who experienced The Electric Company/Success for All Curriculum? 


Using the Program 


29. 


30. 


31. 


32. 


33. 


34. 


35. 


What is your role in delivering The Electric Company/Success for All 
Curriculum? 


How easy is it to use the materials? 


Did you receive training in use of The Electric Company/Success for All 
Curriculum? Was it helpful, and if so, in what way? Would any additional training 
or support have been helpful? 


What is the content focus of The Electric Company/Success for All Curriculum? 
Was this focus helpful to your students? WHY! or WHY! not? 


"Are any materials distributed to parents for use at home? What is the purpose 
of these materials? 


What media or technology (Interactive white board, computers, projectors, etc.) 
did you use to implement the program? Have you used them before for 
teaching? How do you feel they worked for you and your students? 


Describe any challenges you faced in implementing The Electric 
Company/Success for All Curriculum. Do you have any recommendations for 
avoiding or addressing these challenges in the future? 


Reflecting on the Program 


1. 


Overall, how have students responded to The Electric Company/Success for All 
Curriculum? Did different technologies (for example: computer programs, video 
or whiteboards) generate different responses? Do you believe your students 
benefitted from participating? In what ways, if any? How can you tell? 


Has the program benefitted you in any way? Did it meet your needs? If so, how? 
If not, WHY! not? 


Based on your experience with The Electric Company/Success for All 
Curriculum, would you do it again? Do you think it would be useful for you or 
your students to do it again? WHY! or WHY! not? 


How did you like using The Electric Company/Success for All Curriculum? How 
did the children like using or learning with them? 


Did you find The Electric Company/Success for All Curriculum to be unique, 
valuable, or innovative? If so, how? If not, WHY! not? 
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6. What feedback did you receive about The Electric Company/Success for All 
Curriculum from students, teachers, and/or parents? Any memorable stories? 


7. Have your students used interactive whiteboards for learning purposes before? 


8. How well do The Electric Company/Success for All Curriculum fit in with your 
Success for All lessons? Does the program meet any of your students’ needs? 
What needs were met, how were they met, or WHY! weren’t they met? 


9. Is there anything else you'd like to tell us about your experience using the 
program? 


Thank you for your time. 
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The Electric Company School’s Initiative Curriculum Parent Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Parent Interview Protocol 


This protocol is appropriate to use with parents, whose children have been exposed to 
producer’s materials/ activities. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about The Electric Company/Success for All Curriculum. I 
work for the Education Development Center as an evaluator for this project. As part 
of the evaluation, we are interviewing the parents of children who have The Electric 
Company/Success for All Curriculum would like to hear what you think about this 
activity. This interview will take about one 30 minutes. 

Everything you tell us today will be confidential and we will not connect your 
name with anything you say. At any time, please feel free to ask me any questions. 
You can also stop the interview at any time for any reason. 

Please remember we want to know what you think and feel and that there are 
no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we've heard, as we write up our reports. 


Exposure to Outreach Efforts: Technological Enhancements 


1. Is this your child’s/your first time participating in TEC classroom 
activities /lessons? Have you participated in other activities like this one? 
(If so, how many?) 

2. How did you hear about this The Electric Company Curriculum? 

3. Is there anything that caught your attention? 
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8. 
9. 


What does your child think of the program (e.g.: fun, boring, wants to go 
home, wants to do it again, etc)? Do you think he/she liked it? Do you 
think he or she learned anything, and if so, what? 

Have you heard of any activities or lessons that have to do with learning 
to read? If so, what did you hear? Which activities/lessons? 

Have you heard of anything that surprised you about TEC 

activities /lessons? 

Did you see any activities that you or your child might continue to do at 
home? If so, which ones? 

Do you think your child has learned anything from his/her participation? 
Were you familiar with the (computer games, white board, other 
technology) used during TEC activities /lessons? 


10. Where do you look to find out more about programs and activities for 


young children? 


11. Is there anything else you would like to tell us? 
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WordWorld’s eBook Producer Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Producer Interview Protocol 
This protocol is appropriate to use with producers involved in developing the 
WordWorld eBooks. 


Introductory Script 

Thank you for agreeing to do this interview. My name is [NAME], and I'll be 
talking with you today about technological outreach materials/activities that are 
part of Ready to Learn. | work for EDC and we are the evaluators for the Ready to 
Learn Supplemental Outreach. As part of the evaluation, we are conducting 
interviews with producers involved in creating WordWorld’s ebooks. The purpose 
of this interview is to learn specifics about your involvement in developing these 
ebooks. This interview will take about one 1 hour. 

Everything you tell us today will be confidential. To protect your privacy, we 
won't connect your name with anything that you say. At any time during our 
conversation, please feel free to interrupt me if you have any questions or if you 
would rather not answer any specific question. You can also stop the interview at 
any time for any reason. 

Please remember we want to know your opinion about the questions we are 
asking and that there are no right or wrong answers. 

Is it OK if I audiotape this interview today? We review our audiotapes to check 
what we've heard, as we write up our reports. 


Background Information on Ready to Learn Technological Outreach 
1. Could you describe the WordWorld eBooks? 
2. What is your role in producing the WordWorld eBooks? 


3. What is the main message you hope to convey with the WordWorld eBooks? 
(Probe for main learning goals.) 


4. What is your content focus for the WordWorld eBooks? 
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5. Could you tell me about how the WordWorld eBooks came to be part of your 
supplemental outreach activities? How do they extend your overall 
dissemination and communication plan? 


Audience for Technological Outreach 


12. For what target audience or audiences did you develop the WordWorld 
eBooks? Probe for whether target audiences include children (including their 
ages), parents, educators, and/or hard to reach populations. 


13. How do you envision the new WordWorld eBooks being used? 
14. What need do you see the WordWorld eBooks fulfilling? 


15. Did you conduct any needs assessment or market surveys to determine the 
needs of your target audience(s) before developing the WordWorld eBooks? 
If so, what are some of the literacy needs for this audience? 


16. What media or technology does the target audience already use? Describe 
what you provide for end users of the WordWorld eBooks. What materials 
are distributed to the audience? 


a. If your target audience is parents, have they used this media or 
technology before? Did you provide training? If yes, what was the 
goal of the training? 


b. If your target audience is children, have they used this media or 
technology before? Did you provide any instructions for use? If yes, 
what was the goal of that instruction? 


17. Do adults or kids enjoy using the WordWorld eBooks? 


18. In your view, do the WordWorld eBooks successfully reach the adult end 
user? How do you know? 


19.In your view, do the WordWorld eBooks successfully reach the child end 
user? How do you know? 


Responses to the Outreach 


1. Do you have a role in disseminating WordWorld eBooks to adult end users? If 
so, what responses do they have to the WordWorld eBooks? 


2. Do you have arole in disseminating WordWorld eBooks to chiuld end users? 
If so, what responses do they have to the WordWorld eBooks? 


3. Describe any formal or informal ways you have of finding out how well 
WordWorld eBooks are working once they’re launched or distributed. 

4. Do you feel that WordWorld eBooks accomplish their goals? What do they do 
well? What would you change if you could? 
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6. 


5. What feedback about the WordWorld eBooks have you received from 
parents, caregivers or others (ie: partner organizations that distribute the 
ebooks)? 


Is there anything else you'd like to tell us? (Prompt: about the WordWorld 
eBooks and supplemental outreach award?) 


Thank you for your time. 
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WordWorld’s eBook Parent Interview Protocol 


Date: Lead Interviewer: 
Producer: Interviewee Name: 
City: Title of Interviewee: 
Names/roles of others present, if any: Role of Interviewee: 


Artifacts collected: 


Parent Interview Protocol 


This protocol is appropriate to use with parents, whose children have been exposed to 
Word World eBooks. 


Introductory Script 

Introductory Script 

Thank you for agreeing to do this interview. My name is (our names are) [NAME], and we'll 
be talking with you today about a development version of the Word World eBook. The 
eBook we will be showing you is a work in progress, at the “alpha” stage, which means that 
it has a number of bugs that still need to be fixed. We are asking parents with young 
children to help us to find out what parts of the eBook are interesting and fun. This will help 
us create a better, finished eBook. Because the eBook is not yet finished, there may be some 
interactive parts (pictures and words that move around) that are not yet highlighted, and 
there may be other parts that don’t work as well as they should. This is part of the process 
of testing an eBook before it is finished, and we hope you will be patient with these 
interruptions because we need your comments to help create the best eBook possible. This 
interview will take about 30 minutes. 


We work for the Education Development Center as an evaluator for this project. Everything 
you tell us today will be confidential and we will not connect your name with anything you 
say. At any time, please feel free to ask me any questions. You can also stop the interview at 
any time for any reason. 


Please remember we want to know what you think and feel and that there are no right or 
wrong answers. 


Is it OK if I audiotape this interview today? We review our audiotapes to check what we’ve 
heard, as we write up our reports. 


Exposure to Outreach Efforts: Technological Enhancements 
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12. Is this your child’s/your first time using Word World eBooks? Have you used 
books like this one before? If so, how many? 


13. How did you hear about the Word World eBooks? 
14. WHY! did you decide to participate in today’s Word World eBooks activity? 


15. What does your child think of the Word World eBook (probe: level of enjoyment, 
engagement, reaction while reading.)? Do you think he/she liked it? 


16. Do you think your child has learned anything using the Word World eBook? If so, 
what? (probe: vocabulary, letters, words, tracking, page turning) 


6) If this is not your first time using Word World eBooks: 


a. Howis the new Word World eBook different from the old ones? 


7) Were you familiar with the technology used for the Word World eBooks (e.g., 
laptop, desktop, CD-ROM)? Did they seem easy to use? WHY! or WHY! not? 


8) What did you like best about the Word World eBook? What did you like least 
about it? 


9) Would you use Word World eBooks with your child again? If so, how would 
you use them? If not, WHY! not? 


10)Where do you look to find out more about educational activities and 
materials for young children? 


11) If you could recommend that changes be made to the Word World eBooks, 
what changes would you recommend? (probe: story, navigation, placement 
of activity, text, audio) 


12) Is there anything else you would like to tell us about the WordWorld eBooks? 


Thank you for your time. 
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Appendix C: Observations at Site Visits and Pilot eBook 
Presentation 


Observation of The Electric Company School’s Initiative Curriculum 
Brief Description of Event/Activity 


A school site visit was organized in conjunction with the Success for All reading coach. 
The purpose of the site visit was for researchers to observe teachers delivering 
phoneme lessons from with the Success for All Curriculum incorporating videos and 
games from The Electric Company. Furthermore, teachers were later interviewed on 
their experience delivering The Electric Company School’s Initiative Curriculum and 
perceived student response. 


The target audience for The Electric Company School’s Initiative Curriculum is a 
classroom environment where students were being taught phoneme concepts that 


aligned with SFA’s Fast Track Phonics’ lessons. Two 1* grade classrooms were observed 


were students were already exposed to the SFA program structure and teachers were 
familiar with the SFA Fast Track Phonics model. Teachers delivered the curriculum 


borrowing from their knowledge, previous experience inside the classroom, and traini 


provided by SFA and Sesame Workshop The Electric Company School’s Initiative 
Curriculum. In both classrooms there were five African American students, and one 
white student. The teachers were both white. 


The following is a diagram of the classroom setting (see Figure 4): 


ng 
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Figure 4: The Electric Company School’s Initiative Curriculum classroom observation 
diagram. 


Prior to beginning the lesson, the teacher voiced her concern that the assistant teacher 
was missing and that the class would not run as smoothly as it usually does. Although it 
was a Small group of students, and while the teacher displayed a strong set of classroom 
management skills, such as addressing disruptive behavior and interruptions, it was 
evident that the classroom environment would have benefitted from additional staff. 
During the observation, one student had to be removed from the classroom because of 
his behavior. One student with autism and a speech impediment required additional 
assistance to complete the scheduled activities. The teacher used “Pal Bucks” for 
supporting good behavior and used a point system (1-3) for identifying behavioral 
issues. Students shared stories from home with the teacher. Overall, the teacher was 
busy managing the classroom environment and delivering the lesson to the students. 


The teacher delivered lesson 13 /ow/ (cow) 1/1 from The Electric Company School’s 
Initiative Curriculum. As planned, the lesson was integrated with the Fast Track Phonics 
portion of the SFA Curriculum. The Electric Company School’s Initiative Curriculum was 
approximately 45 minutes of the 90 minutes literacy period. 
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A computer’s hard drive executed The Electric Company School’s Initiative Curriculum. 
The teacher later stated that she did not have access to the PBS Kids website, where the 
curriculum and other materials could be found online. 


The teacher used the following hardware: desktop PC, projector, and interactive white 
board. The teacher implemented the lesson using the SFA’s Fast Track Phonics Booklet 
and SFA’s literacy curriculum in conjunction with The Electric Company/Success for All 
product. 


The following is an outline of the scheduled activities: 
* Collection of homework assignments (10 Minutes) 


¢ Discussion of the day’s activities (1 Minute) 

e “Say It Fast” lesson, Interactive White Board (10 Minutes) 
¢ Word Level Blending Activity (2 Minutes) 

¢ “Say What” Video (2 Minutes) 

e “Word Transformer” Game (6 Minutes) 

e “Stretch and Read” Video (3 Minutes) 

¢ Journal Writing (12 Minutes) 

e “A Farm in China,” Book Discussion (10 Minutes) 
¢ Book Vocabulary (12 Minutes) 

e “A Farm in China,” Book Reading (10 Minutes) 

¢ Story Test and Writing Assignments (15 Minutes) 


The teacher called on students to provide a “7-up sentence,” a sentence with seven 
words including the word on the screen. Students were quick to respond and many 
raised their hands to participate. Additionally, the teacher used The Electric Company 
videos as a review for the previous lesson or as an accompaniment for the next activity. 
Transitions between activities were smooth and the interactive white board responded 
instantaneously and without any technological glitches or challenges. Students were 
eager to participate and they were given several opportunities to interact with the 
lessons. 


Next, students had the opportunity to play The Electric Company School’s Initiative 
Curriculum games. Some students had the opportunity to play the games, but not all. In 
general, The Electric Company/Success for All Curriculum games allow for only one 
student to interact with the screen at atime. The classroom teacher managed game 
play by continuing with the next child in line from the previous game. The teacher 
announced, “The students who did not get a chance to play will be the first ones to use 
the board for the next class.” During game play, students clicked their selections on the 
screen over and over again with the pen, often hitting the wrong spot and not correctly 
dragging letters with the whiteboard pen. Students were observed talking out loud 
while playing the game. One student responded at the conclusion of the “Say What” 
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game by stating that “He always wins!” Another student asked “How come we always 
get at 100 points and how come we do not get big points, 102,103?” 


While the lesson moved at a fast pace, the teacher managed The Electric Company 
School’s Initiative Curriculum and use of the interactive white board. The Electric 
Company parts of the lesson were interweaved with SFA’s Fast Track Phonics activities 
and seemed to have strengthened the structure of the curriculum. This was evident in 
the transitions between activities and the student engagement from one activity to the 
next. 


69 


Observation of Super WHY! Summer Reading Camps 
Brief Description of Event/Activity 


Researchers conducted a site visit to observe Super WHY! Summer Reading Camps 
integrated with Super WHY! Computer games in a school district where multiple sites 
were implementing the program for one week as part of the state’s Volunteer Universal 
Pre-Kindergarten program. The program is run through the district’s Title | office and 
generally serves students who have not yet participated in a Pre-Kindergarten program, 
or who have been identified as needing additional classroom experience to prepare 
them for the coming school year. 


Three observers spent time in two classrooms in a single school. The students in each 
class were 4 and 5 years old, African Americans, and one Caucasian student. One 
teacher was Caucasian and the other African American. Camp Facilitators in each room 
were classroom teachers during the regular school year and this was their first time 
teaching the Super WHY! Summer Reading Camp. Following the observation teachers 
from both classrooms participated in an interview. 


Below is a general diagram of the classrooms observed (see Figure 5): 


Figure 5: Super WHY! Summer Reading Camp observation diagram. 
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Every teacher facilitating a Super WHY! Summer Reading Camp receives the same 
materials including a scripted curriculum guide. At the beginning of the observation 
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teachers lead discussions with their students about word families (‘—all and ‘-ig’). The 
teachers wrote a list of words as children named them (ball, wall, call, etc) and the 
students practiced reading the words. Teachers used a lot of repetition in this activity by 
writing the words in the word family, reading them out loud to the students, and by 
having students take turns reading the words. 


In each of the two classrooms, several students asked their teacher if they were going to 
watch Super WHY!. Following the scripted curriculum, the teachers introduced the 
Super WHY! video, The Three Little Pigs, to the students. The teachers reviewed each of 
the main characters they had been introduced to already and their related super 
powers. During the viewing, students interacted with the video, singing along with the 
show, answering questions, doing the movements encouraged by the characters, and 
reciting the characters’ lines. The same episode was shown each day of the camp, so 
students were familiar with the story. 


While the students were watching the video, the teachers loaded Super WHY! games 
onto desktop computers in the classroom. In one classroom, the game’s audio was 
repeating a line inviting players to begin to play. In this same room, the teacher and 
another adult were talking while the children watched the video. Some children were 
distracted, listening to the adults’ conversation and not to the and not to the video. 
When the video had finished, teachers reviewed the story with the students, asking 
them what happened and reviewing the characters’ super powers again. 


Next, students received a Super WHY! character mask to wear. Teachers and assistants 
put the masks on the children’s faces. Teachers introduced the next activity, playing a 
computer game. In each room students gathered around the teacher as she introduced 
the game. The teacher was in control of the mouse while verbalizing instructions for 
game play. The students advised the teacher where to move the Super Why, Why flyer 
on the screen (“go faster, go up!”). The Why flyer has to drive over letters that appear 
on the screen and once accumulated, the system makes a word that fills in a blank in a 
short sentence. One teacher informed her students that they would have ten minutes 
after lunch to play the game, but that they couldn’t play it now. 


Following the game, students did a drawing and writing activity. In one classroom, 
students were instructed to “Draw a story”. In the other room they were told, “Today 
we are going to draw pictures and tell a story.” In the latter class, the teacher gave 
detailed instructions about what the students were going to do. In both classrooms, 
teachers went around and talked to students about their work. They helped the 
students write about the pictures they drew. 


One child was observed playing the computer game after he finished his drawing. Upon 
completing of his picture, a student gave the paper to the teacher and asked permission 
to use the computer. The teacher allowed and the student sat down in front of the desk 
top computer. He put on head phones and began to play the game. For a few minutes, 
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until the other students finished up their work, he played the Super WHY! Why Flyer 
Adventure game independently. When all of the students in that classroom had 
finished working, the teacher asked the boy to stop playing the computer game and 
rejoin the group as they shared their work with each other. 


The teachers led students to the next activity “Pin the tail on the word”. The teacher 
read a sentence with a blank space and the students had to choose the word that best 
fit into the sentence (ex: big, bad / little, scared). The students were active during the 
discussion, each having a turn to choose a word. One of the teachers was constantly 
revisiting the curriculum guide. She explained that she was up late the night before and 
did not have time to read it ahead of time. 


The final activity observed in one of the classrooms was called the “Super WHY! 
opposite relay”. Words are taped on to a giant cut out of the character Super WHY!. The 
students stood at the opposite end of the rug from the cut out, each holding a word 
written on a card. One student cried because he didn’t have a turn, though the teacher 
had just begun calling names. Other students were fidgeting while they waited. They 
took turns ‘running’ to the cut out to find the opposite word to the one they had. 


The teachers both followed the curriculum, with some variation in implementation. 
Overall the students in both classrooms were engaged with the video viewing and the 
activities. As described above, only one student was observed playing the computer 
game independently. 
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Summary Observation of the initial Presentation of the Word World eBook pilot 
Description (or diagram) of Setting 


Figure 6: Initial presentation of the WordWorld eBook Pilot observation diagram. 


Brief Description of Event/Activity 


A site visit to observe parents and children using a new Word World eBook was 
conducted in the library of a school during an afterschool program (see Figure 6). In 
cooperation with a local Ready to Learn Outreach Station, EDC’s researchers observed 
and interviewed 4 parents of children ages 3, 4,5, and 6. One parent/child pair was 
African American, two were Hispanic, and one N/A based on observation. Each of the 
Hispanic parents were interviewed in Spanish, and the other two in English. 


Parents/caregivers arrived sporadically at the school and were directed to the “Library” 
room. Four laptop computer stations were set up on two tables. Each laptop was 
connected to the local network via a wireless connection, and consisted of a USB 
connected mouse, mouse pad, and headphones. The Word World eBook web link was 
accessed using a Firefox web browser. The cursor size was enlarged (approximately 
three times its size) using the operating system’s universal access settings. In addition, 
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the web browser window was zoomec-in to expand the eBook without cropping its 
width or length. 


Upon arriving, parent/caregiver-child pairs were welcomed to the event and introduced 
to the researchers. They were then briefed on the planned activities. Parents signed a 
consent form agreeing to participate, and each received a $40 gift card as a thank you 
for their participation. Parents were situated at a laptop computer next to their child. 
Researchers introduced the WordWorld eBook to each pair, using the following script in 
their introduction: 

The eBook we will be showing you is a work in progress, at the “alpha” stage, 
which means that it has a number of bugs that still need to be fixed. We are asking 
parents with young children to help us to find out what parts of the eBook are 
interesting and fun. This will help us create a better, finished eBook. Because the eBook 
is not yet finished, there may be some interactive parts (pictures and words that move 
around) that are not yet highlighted, and there may be other parts that don’t work as 
well as they should. This is part of the process of testing an eBook before it is finished, 
and we hope you will be patient with these interruptions because we need your 
comments to help create the best eBook possible. 


Researchers encouraged parents to look for interactive elements in the e-Book, which 
can be recognized by the duck wing, and modeled viewing using the first two pages of 
Totally Terrific Duck. First, researchers showed the parent/child pair how to click the 
succession of 3 pictures of Duck in the picture frame. They were also shown how to 
click on “BARN” to see letters spread apart and hear the voiceover “Barn!”. The next 
page of the book included a word build of the word T-I-E. Parents and children were 
reminded to listen for instructions to build the word. Lastly, researchers demonstrated 
how to click on ‘BUG’ to see letters spread apart and hear the voiceover “Bug!” 
Parent/Caregivers were directed to read the eBook “Totally Terrific Duck” together with 
their child and inform the researcher when they were finished. 


Each parent/caregiver was individually interviewed after the reading, while the child 
was given the choice to re-read the Word World eBook, choose a bound book to read, 
or play with toys in the room. Two interviews were conducted in Spanish with the 
Hispanic parents, and two interviews were in English. 


Summary of Parent and Child responses 
Use of eBook 
1. Overall, the eBook was well-liked by parents who stated: 
a. * “[My son] liked it very much, the colors of the little Duck were very 
attractive to him and the story line.” 
b. * “It teaches you, it also helps [my son] get into using [technology] and 
get out of the daily routine of learning.” 
c. * “The story is short but adequate for the kids of this age. And it is 
interesting, | like everything about it.” 
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2. Children navigated through the book with some/minimal assistance from their 
parent/caregiver. Parents stated: 

a. *“[My son] controlled it. | would guide him and he would move the 
mouse to change the page...Only the first time did | tell him what he was 
supposed to do, and then he went on by himself.” 

b. ”[My daughter] did most of it herself. | tried to help her manage the little 
mouse. | liked it highlighting in big when you came to something you 
could click on. She looked for it.” 

3. Two children reread the eBook a second time, at a faster pace, while their 
parent/caregiver was interviewed. Parents stated: 

a. “She said, Mommy | want to read it again.” 

b. “lf she had it at home she would read it again and again.” 

4. The eBook went at a good pace for the parent/child pairs and was scaffolded. 
Even when the pace seemed too quick for a child, the parent or child could click 
on the words to have them repeated. One parent explained: 

a. “It wasn’t too fast. It helped that [my daughter] could go back through it, 
you know, even if it did go too fast she was able to go back and click on 
the words.” 


Reaction to eBook 
5. All parents mentioned this is the first time they had been exposed to eBooks, 
stating: 
a. “This is my first one.” 
b. “It is only my first time.” 
6. Children seemed to enjoy the interactivity, including clicking on the words, and 
seeing animations. 
a. Two parents suggested that the characters were very likeable. 
i. * “[My son] liked the Duck’s hat, its tie along with its animation 
and colors.” 
ji. “[My daughter] liked the duck.” 
7. Parents expressed happiness that their child was learning from the eBook (see 
skills mentioned below). 
a. Parents thought the eBook was colorful and children were engaged. 
i. “liked that it’s colorful.” 
ii. “It reads itself like TV. [My daughter] can see it and understand.” 
b. One parent mentioned that her child would choose to read an eBook 
over a real book because it is more fun. 
i. “It helps them participate in reading, instead of just reading a 
book.” 
ii. “It reads to her.” 
8. Parts of the story most liked: Duck’s expressions changing in the mirror, word 
morphs 
a. “The mirror part is good.” 
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b. “She liked the duck, when you clicked on it and it changed to the sad 
face.“ 

c. “She likes when [the words] transform.” 

9. One parent, of 4-year-old child, mentioned she did not feel eBooks had enough 
variety, like traditional print books. 

a. *“Achild can hold the traditional print book in their hands. If they don’t 
like it, they can grab another one....but in a center, one can have a variety 
of books available, which | don’t think its like that with something on the 
computer.” 

10. All parents mentioned they would use the eBook again. 

a. * “As long as one guides [my son] and says ‘Look, we are going to do this 
program.’ And if it interests him, as long as it’s creative.” 

11. One parent mentioned it supported a positive use of technology, rather than 
watching inappropriate videos/movies in children’s spare time. 

a. *“l would recommend the eBook, rather than using other movies that are 
inappropriate for them. Instead they can enter and read the books that 
are found on the PBS site or other programs on the computer. Give them 
more information about these programs.” 

12. Two parents of dual-language learners asked researchers to return to present 
more eBooks. 

a. * “Please return so we can read some more books... Are you going to send 
us more information on where to find them?” 


Participants’ Expressed Views of Educational Media Tools: 
¢ The parent of a 3-Year-old dual language learner mentioned that her child 
already has experience with computers, and enjoys online educational software. 

o * “My older daughter shows him some programs and he really likes it on 
the computer. He controls the mouse and uses it. Also here in the center, 
he plays on the computer.” 

¢ Both Spanish-speaking parents use Curriculo Creativo (Creative Curriculum) as a 
starting point for finding more educational tools. 

o *“l goto the internet for things. | enter the Creative Curriculum. It is a 
program we use here at the school. | look for it in Spanish, and from there 
| begin to get ideas. They have activities and procedures and materials.” 

* Two Spanish-speaking parents, of dual language learners, mentioned the older 
siblings in the family are the main source for bringing educational 
materials/activities into the home. Older siblings use their own experience as 
resources 

o * “My older daughter has some programs she uses at the school where 
she is at and it has a program for younger children and she enters the 
webpage for him and he learns colors, words, the basics. And he likes it a 
lot, he entertains himself.” 

¢ Two parents mentioned they trust that producers will implement the best 
pedagogy and educational practices into their product. 
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o * “Tome, everything that is educational is okay with me. Because you are 
not going to put something that is not adequate for learning.” 
Two parents of dual-language learners said they felt they constantly ask teachers 
from the after-school program for more educational resources but are not 
provided with enough tools or attention to their child. 
Two of the four parents mentioned they own a computer at home. 
o One parent mentioned she owned other electronic educational tools. 


Parents’ Statements about Skills Addressed by eBook 
13. Phonemic Awareness: 
a. Choosing letters to put together to make a specific word (ex: t-i-e) 
i. “It is good to show how to write a word.” 
ii. “like the t-i-e part.” 
14. Phonics 


a. The connection between sounds and letters — ‘t’ sound is pronounced as 
the letter ‘t’ inflates 


i. “Seeing the words put together, knowing that each letter has a 
sound. “ 
ii. “[The eBook] is good for pronunciation.” 

b. The understanding that sounds blend together to make a word: Parents 
liked that the words put together (ex: t-i-e) were read separating the 
sounds 

i. “She knows letter sounds. Putting the sounds together is where we 


are going next. It is good for her, seeing the word put together, 
knowing that each part has a sound.” 
15. Concepts of Print 


a. Tracking (words highlighted) 
i. Directionality: they can see the left to right movement of reading 
a. “She can see it and hear it and so she can follow it.” 


b. “They highlight the word and she can understand 
[the story].” 


b. One-to-one correspondence: they can see and hear the word at the same 
time- Parents liked that the words could be clicked and repeated 
i. “It helped that she could go back through and click on the words.” 
c. Not all parents know how to read a book in a way that teaches literacy 
skills, like pointing to words as they read. The eBook does this for them as 
it highlights 
i. “if she picks up a regular book she has to get an adult to read it to 


her and adults are going to just read it. They don’t know to point 
to the words.” 


16. General reading skills and engagement 
a. Overall confidence as a reader. Reading the eBook gives the child a 
chance to recognize words they know and click them to hear the word 
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i. “We’ve been working on site words and she can click on a word 
she knows and hear it. She feels more confident.” 
b. Understanding that words have meaning 
i. “The words actually mean something and it shows what they 
mean when they are formed, and then children can associate the 
word with what it means.” 
c. Vocabulary —the word “awry” is a new word 
i. “l' was surprised when it said “went awry” rather than ‘going 
crazy’ or something. They used another word. | wished that there 
was more of those words.” 


Suggestions from parents: 
1. All parents mentioned this is the first time they were presented with an eBook 
and were not able to compare its educational value to another eBook. 
2. Parents said they would prefer to receive a CD-ROM of an eBook, rather than a 
link to the internet. Parents stated: 

a. *“l! would prefer the CD, they are more practical....1 have three kids total 
and they all have CDs. My 11 year old daughter helps [my son] out a lot 
with it...Email would work, but you don’t have it. | like the CD. ” 

b. Parents suggested that a CD is better because children can hold it and see 
it, as a reminder 

i. “Something tangible that | can see and remind me.” 

c. Also they stated that a CD seems to be more practical 

i. “CDs always work for me.” 
d. Parents also stated that there was no time for email/internet during the 
day 
i. “We don’t really get online during the day.” 
3. Some parents would like more interactivity on each page 
a. More word morphs and clickable items like the Barn 
i. “lf there were more things she could click on, like the BARN.” 
4. The caregiver guide was not discussed as something parents liked about the 
eBook. 
5. One parent wanted Pig to be introduced as the photographer earlier in the story, 
rather than just sitting in the background, not doing anything. 

a. “The pig is just there in the background. Maybe at the beginning they 

could say something about the pig. Like he is the photographer.” 
6. Two parents suggested making the eBook longer 

a. * “Depending on time, | would like to have it be longer. Yes, so it lasts 
longer.” 

b. “It is too short for me.” 

7. Two parents suggested the book be translated into other languages 

a. * “Not all of us understand. There are a lot of parents who have children 
that come from different countries and they don’t understand the 
[English] language.” 
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b. One parent suggested the eBook provide a handout that includes a 
translated version of the story. 

c. Two Spanish-speaking parents feel a translated copy would also allow the 
adult to learn as they accompany the child. Spanish-speaking parents 
indicated they would like to know what their child is reading (with regard 
to content). 

i. * “Translated, it would also help one understand the other 
language. | follow along, but | don’t understand what is going on. 
The other language is necessary.” 

ii. * “They learn the second language in the school. Maybe | can 
learn a bit at home with them. | would add support for more 
languages.” 

8. Adding more new vocabulary like “awry” that a child doesn’t hear every day 

a. “l would like to see more bigger words, more vocabulary, not just words 

they already hear and know.” 


* Please note, these responses have been translated from Spanish. 
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Appendix D: Summary of Survey Findings 


Summary of Survey Results 


Super WHY! Summer Reading Camp 
Camp Facilitator Survey Findings: 


In June 2010 we distributed a survey of 37 items to Super WHY! Camp Facilitators 
following their weeklong summer reading camp implementation period. Station 
education outreach coordinators, emailed the survey to camp facilitators who were 
provided with a gift card of $20 upon completion on the survey. Of the 86 camp 
facilitators, 23 completed surveys. We speculate that the indirect method of reaching 
out to camp facilitators had a negative impact on the response rate to this survey. 
Because Out of the Blue staff worked with regional education outreach coordinators 
who in turn reached out to individuals in school districts across the country, there was 
no central contact for teachers and therefore no direct way to communicate with 
teachers to ask that they complete the survey. Therefore we suspect that either few 
teachers received information about the survey or few understood its purpose or were 
reminded to complete it. 


Survey Overview 


The survey included a range of questions including 6 that required a yes/no response, 27 
multiple choice and matrix questions, and 5 open-ended questions. 


The questions that elicited a yes/no response asked teachers about their past 
experiences with computers and with the reading camp, if they saw value in the 
addition of the Super WHY! games, and how the games were used in the reading camp 
curriculum. 


Multiple-choice questions aimed to collect information about which games were used 
and liked, how and when students played the games, if the pacing and content was age 
appropriate, and if the games were meeting their needs and those of their students. 


The 4 open ended questions allowed teachers to share their thoughts about the 
educational value of the games, what they liked most and least about the Super WHY! 
games, how students responded to the games, and if the over all needs of their students 
were met. One open ended question asked teachers to describe their past experience 
with computers. A fifth open-ended question was a follow-up to a yes/no question 
about previous use of computers and computer games with students. 

1. What did you like best about the computer games? 

2. What did you like least about the computer games 
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3. How did your students benefit from using the games? (If teacher reported 
they did benefit) 

4. |s there anything else you'd like to tell us about your experience using the 
computer games 

5. Have you used computers or computer games with students in your 
classroom in the past? If yes, please explain how. 


Responses to multiple-choice questions: 


Most of the teachers who completed the survey had experience using computers and 
computer games in the past with students. Only 2 teachers had never used computers in 
their classes in the past (see Figure 7). Of the 21 teachers who had used computers in 
the past, they mainly used computers in their classrooms for learning games and 
educational websites. About an equal amount used computers to help reinforce and 
review skills they were teaching. Five teachers used computers for both instruction and 
as a center in the classroom. The least common ways teachers used computers in the 
past were for test preparation, watching videos, and introducing new topics. 


Figure 7: Facilitators who have used computers or computer games with students in the 
past. 


Facilitators Who Have Used Computers or Computer Games With Students in the 
Past 


Yes 
91% 


Teachers informed us of the ages of the students in their summer reading camp class. 
The majority of the teachers had students ages 4 and 5. There were very few 6 year 
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olds, and even fewer students who were 3. Most of the classes were of mixed ages, and 
only 4 out of 23 classes had children of all ages 3-6. 


We asked teachers to rate their degree of agreement with statements about the 
computer games and their use. The response options were; strongly disagree, disagree, 
neutral, agree, and strongly agree. In general, teachers’ responses about the games 
were positive (see Figure 8). Twenty-two of the 23 teachers agreed with the statement 
“| would be excited about using computer games in the Super WHY! Summer Reading 
Camp again in the future.” The same number of teachers noticed their students 
interacting verbally while playing the computer games, calling out answers or discussing 
with a partner. Twenty-one teachers found it easy to make connections between the 
computer games and the Super WHY! Reading camp curriculum. Twenty-one of 23 
teachers disagreed that they would rather have taught the camp without the games, 
further illustrating their fondness of the games. The most frequent ‘neutral’ responses 
were about what campers did outside of the classroom: talking about the games in 
another setting, or playing the games at home with their parents. It is likely teachers 
were neutral about these statements because they are unsure of students’ activities 
outside of school. Teachers reported that an equal number of campers played the 
games independently as played with others. Seventeen teachers disagreed that their 
students only played games as a whole class. 


Figure 8: Facilitator agreement with statements about Super WHY! web-based 
computer games and the Super WHY! Summer Reading Camp Curriculum. 
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Super WHY Summer Reading Camp Curriculum 


BSsrongly Agree ® Apree ONeutral O Disagree ® Strongly Disagree 


25 


[2] 
| Fi 
| 7 
| 10 
| "2 


ra 


Number of Responses 
— wm 
wo co 


w 


2¢ Es 272% Ey Fa ¢ es r = 
sE Es «aged EEe  3f ge; af 
ee ays E258 $23 sf £5 sad 32 
a 2: so% ste e&% 22 z iss Fe 
23 foe fits 858. g2ag 5s 5358 38 
>fe ef8e Z2€E5 £225 §g8 Ge gs £ 23 
a8 Ecod faa? e p= 7 Sees *8 egies 5 
352 nr grez s§rt te 58 tS. By 
£ b4 se =R ro. re) eoeta 8 a= 
aioe gee Pp Hp Gb one it 
- gE E885 3 Pa abs aS SERE s% 
OOD Bebop ooh ap 
= ae = © Responses 8 ~ es 


82 


When asked about how successfully the computer games fit into the daily schedule, the 
majority of teachers liked the daily schedule of activities, including how the games were 
introduced to students (see Figure 9). Many students had the opportunity to play the 
games more than once during the day, though some teachers said their students only 
played once each day. Twenty-one out of 23 teachers were comfortable with the 
number of adults in the classroom to support the children while they played the games. 
Four classrooms had 3 adults present, and about half of all 23 classrooms had two adult 
supervisors. Several teachers stated that one adult is sufficient to supervise the 
students, as teachers were alone in their classrooms while students were playing the 
games. 


Figure 9: Schedule and Supervision. 


Schedule and Supervision 
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Did the schedule of activities Did campers have more than one Did you have enough adults to 
including introduction of the opportunity during the day to play supervise the campers while they 
computer games work for your the computer games? used the computer games? 
campers? 


Responses 


In addition to teachers reporting how much they agreed with statements about the 
games, they were also asked to report their thoughts about the games in relation to 
how their students used them (see Figure 10). There were three statements that no one 
disagreed with. Teachers overwhelmingly agreed that playing the computer games 
helped students feel more comfortable and familiar with technology. Twenty-one 
teachers agreed that their students enjoyed playing the games, the while two were 
neutral. All teachers agreed that, “the computer games helped to reinforce the skill of 
the day.” About 75% of the teachers thought that overall the instructions for the 5 
games were clear and easy for the campers to understand and that the games were 
appropriately paced. Fewer teachers agreed that the content and words used in the 
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games were at an appropriate level for their students. Overall, teachers were 
comfortable with their knowledge about the computer games with their young 
students. Twenty out of 23 teachers indicated they did not need more training on how 
to use the games. 


Figure 10: Facilitator agreement with statements about the Super WHY! web-based 
games. 
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As part of the Super WHY! reading camp curriculum and schedule, teachers introduced 
the game of the day to their students as a whole class. The curriculum suggested 
allowing other time slots throughout the day for students to play the game that was 
introduced to them, but the exact time would be the teacher’s decision. Other than the 
whole class introduction, it was up to the teacher whether the students played 
independently, in pairs, in groups, or by communicating responses to a teacher. Only 15 
teachers reported that their students played the game as a whole group, given that the 
curriculum stated they should introduce the games to the whole class at once (see 
Figure 11). The next most common way students played the games was independently 
in 14 classrooms. Students played the games in pairs in 10 of the classrooms. The least 
common ways students played were in groups and with teachers. When explaining the 
‘other’ way her students played the games one teacher wrote, “The first day | sat with 
each pair of students and helped them through.” 
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Figure 11: How campers played the computer games. 
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During the week of the Super WHY! reading camp, one computer game was introduced 
each day. Each of the five games related to a character in the Super WHY! show, and 
each character had a literacy-based super power. The games are: 

¢ Storybook Creator 

¢ Alpha Pig’s Alpha Blocks 

¢ Wonder Red Freeze Dance Rhyming 

¢ Princess Presto’s Golden Crown Spelling Bee 

¢ WHY! Flyer Adventure 


Number of — 


Students played Alpha Pig’s Alpha Bricks and Wonder Red’s Freeze Dance Rhyming the 
most (21 out of 23 classrooms) (see Figure 12). Students also often played Princess 
Presto’s Golden Crown Spelling Bee and the WHY! Flyer Adventure. The computer game 
Storybook Creator was least commonly played, as only 9 teachers reported its use. 
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Figure 12: Games played by campers. 
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Of the five games, Alpha Pig’s Alpha Bricks game elicited the most positive response 
from students (see Figure 13). About half of the teachers reported their students 
responding positively to the WHY! Flyer Adventure game. Only 1 teacher thought the 
Storybook Creator game got the most positive response. About a third of the teachers 
reported positive reactions to Wonder Red’s Freeze Dance Rhyming and Princess 
Presto’s Golden Crown Spelling Bee, 7 and 8 classrooms respectively. In the open ended 
responses a few teachers noted frustrations with the last two games mentioned. 
Teachers thought that the skill level was too high for their students. Therefore, Alpha 
Pig’s Alpha Adventure game might be more appropriate for the literacy level of these 
young students as it focuses on letters rather than words. 


86 


Figure 13: Games that elicited the most positive response from students. 
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Of the 23 teachers who responded to the survey, about half (12) of them had taught the 
Super WHY! Summer Reading Camp in the past. The 12 teachers who had taught the 
camp before answered questions that compared the past camp to the new curriculum 
integrated with the computer games (see Figure 14). None of these teachers thought 
that compared to the original camp student participation, attention and laughter was 
“less than usual” with the addition of the games. Six of these returning teachers 
reported that their campers’ participation was “about the same as usual” as the other 6 
returning teachers thought their campers’ participation was “more than usual”. During 
the day 8 of 12 teachers reported that campers’ attention was “about the same as 
usual”. Similarly, about half of these teachers heard their students laughing “about the 
same as usual”. Exactly half of the teachers who responded were as interested in the 
lessons before and after the addition of the computer games, and half were more 
interested after the addition of the games to the camp. 
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Figure 14: Differences between the original Super WHY! reading camp without 
computer games and the new camp with computer games. 
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Overall the information collected from this survey conveyed a generally positive 
response from the facilitators of the Super WHY! summer reading camp integrated with 
computer games. All but one teacher thought their students benefitted in some way 
from the games. The one teacher who did not feel her students benefitted from the 
games was not able to use them because of technical problems. In addition to replying 
‘No’ to the survey question, “Do you think your students benefitted from the computer 
games?” she explained WHY! she responded this way, “We could not use them. | think it 
will be a good addition to the camp when the program is working well.” Therefore all of 
the teachers saw benefits of the games for their student, or saw the potential for these 
benefits. 


Responses to open-ended questions: 


What was most-liked about the games: 

About half of the 23 teachers most liked seeing their students so engaged with the 
games and the amount of laughter and excitement students expressed as they played. 
One teacher commented enthusiastically, “/ loved the stimuli. They were colorful, there 
was zipping, and popping. The kids loved the way they were able to interact with each 
character and sing their special songs especially with Wonder Red (all-ball- call... so 
catchy!)”. Another mentioned, “The kids LOVED the chance to play on the computers. 
They took turns with partners and were eager to participate. The skill level-appropriate 
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games reinforced the day's lesson.” 


In addition to teachers liking how the games supported literacy skills in a fun and 
engaging way, three teachers thought that it was useful for students to have experience 
using computers. “Some of my kids have never played a computer game or touched a 
computer. So the simple fact that they were able to actually play, laugh, learn and have 
fun was the best part.” Three teachers also thought the games were age appropriate for 
students. This could explain WHY! more than half of teachers noted their students 
would work independently at the computers. This could also be a result of the whole- 
class introduction to the games, which 13 teachers did with their class. 


Fourteen out of 23 teachers who responded liked how the games supported students’ 
literacy skill development. “They seemed to progress throughout the week” said one 
teacher. “The children love to use the computer and | was happy that it reinforced the 
goals of the day.” “They were able to put the new skills they had learned into practice 
immediately.” 


What was liked least about the games: 

When asked what they disliked most about the program, teachers most often identified 
the problems they had using the technology. About half of the teachers found loading 
the games to be frustrating. They said it took a long time to load the games, and some 
of the games rarely worked at all. A teacher explained her frustrations, “The games at 
times kept freezing or never downloaded. We wasted a lot of time waiting for them to 
load.” One teacher mentioned other problems with the technology, “One wrong click 
from the child, caused them to easily be diverted to another page. Also any game that 
depended on listening was difficult because we did not have head phones for all children 
and the sound was muffled at times that | could not even understand if the computer 
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was saying "p" or "t" for example.” 


In addition, seven teachers reported that some of the games were too difficult for their 
students, in particular three teachers said that Wonder Red Freeze Dance Rhyming was 
the most challenging and inappropriate for her students. One teacher noted, “Some of 
them were too hard for this age group because they couldn't read yet so for example one 
of them like Wonder Red's fun, bun, run where you had to know what the word said to 
play independently was too hard for them.” One teacher also commented on the 
Princess Presto Spelling Bee game, “being that we are working with small children, they 
love immediate feedback. With that being said, do we really have to get four answers for 
one jewel to appear in the crown each time? It just made a seemingly fun game drag on 
and each time students thought that they would see a complete crown and did not.” 


Benefits of the games: 

Of the 23 teachers who responded to the survey, all but one said they thought their 
students benefitted from playing the games. Two teachers reported students’ test 
scores went up. One teacher stated: “Having taught this camp for the last several years, 
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| noticed better results on the post tests.” The other wrote, ”/ think the games gave 
students more review and practice of skills and | thought my students tested well”. 
Teachers were aware that students were having positive experiences with the games 
because of the types of responses they observed. A teacher explained that she knew her 
students were benefitting from the games because “the kids were actively and excitedly 
discussing the task at hand and laughing often.” Another teacher noticed her students 
were having a positive experience because of “the conversations that were taking place 
between them and the excitement they showed by yelling out answers.” The one teacher 
who did not think her children benefitted from the games was not able to use them 
because of technical problems. She did note however, that she thought the games 
would be beneficial if they worked for her. 


Recommendations from Teachers: 


In their final comments about the Super WHY! computer games, some teachers 
suggested ways to improve the games and their use. As explained above, there were 
some feelings that the skill level of some games was too difficult. Teachers suggested 
making the skill level more appropriate: 


“The rhyming games were too hard. It would be helpful if the students could see 
and hear the words because they could not read the words.” 


“In Princess Presto Golden Spelling Crown the 3-5 letter words were great until words 
like pumpkin and basket were introduced. Some of the choices the campers were given 
were illogical. There were 2 random letters together, not even blends, to choose.” 


In order to make the games a little more age-appropriate for the younger students it 
was suggested that the games “use more praise in the game and give feedback that 
even the student can tell what they have done.” In terms of improving ease of navigation 
for little hands , “make buttons bigger that say start again, or simple words like Go/green 
and red/stop.” 


It would be helpful for teachers to know how their students are performing on the 
games. It is important for teachers to know where their students’ strengths are, and 
where they need to improve. A teacher noted that “it would be nice if | as the teacher 
could see what the student played and how successful they were.” 


Many teachers reported technical difficulties while playing the games, especially during 
the loading process. Teachers were frustrated with the amount of time it took to load 
the games. One teacher suggested, “/t would be nice to have them ona disk too, in case 


there was a problem with the internet connection. 


One teacher suggested to “consider the ELL students”. 
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The Electric Company School’s Initiative Curriculum Teacher Survey Summary 


Researchers developed and disseminated an online survey for The Electric Company 
School’s Initiative Curriculum teachers regarding their experience in delivering the 
Success for All Curriculum with integrated The Electric Company videos and games. 
While a total 25 responses were submitted via the online survey, 23 teachers completed 
the survey from start to finish. Some teachers skipped some questions, depending on 
their use of the curriculum and voluntary choice to answer specific questions. 


The survey included 25 questions, including open-ended and multiple-choice questions. 
Teachers’ Background and Use of The Electric Company School’s Initiative Curriculum 


Of the 23 surveys completed by teachers, 21 reported that they delivered more than 6 
lessons from The Electric Company School’s Initiative Curriculum. A majority of the 
teachers, 16, stated that they delivered about 11-15 lessons. Three teachers delivered 
less than 5 lessons. 


Findings suggest that 13 out of the 22 teachers have watched at least one full episode of 
The Electric Company at the conclusion of their participation with the program. 


Using the technology 
Our findings from the online teacher survey, about teacher’s experience with The 
Electric Company School’s Initiative Curriculum, shows the following: 
* Approximately 64% of teachers reported that they worked with students at or 
above the SFA Phonics Reading Level. 
* approximately 75% of The Electric Company/Success for All teachers have 
previous experience using an interactive white board in the classroom. 
* 90% of teachers reported that they have used videos/movies, or computer 
games prior to using The Electric Company School’s Initiative Curriculum. 
*17 out of 20 SFA teachers reported that they had previously used some sort of 
online educational resource. 
* Two teachers out of 24, feel they could have benefitted from more training on 
how to use the technology to deliver The Electric Company School’s Initiative 
Curriculum. 


The Electric Company School’s Initiative Curriculum Content 

Teachers were asked to rate their degree of agreement for a set of statements regarding 
their experience with The Electric Company School’s Initiative Curriculum on a five-level 
Likert scale. Our findings show that about half of the 24 teachers felt the SFA lessons 
with TEC content were well organized and easy to implement. Paired with the open- 
ended response of what teachers liked least about The Electric Company School’s 
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Initiative Curriculum, these findings show that a large group of teachers would like to 
see improvement in way The Electric Company content is structured with SFA lessons. 
For example, teachers would like better alignment between TEC media and the 
appropriate grade level of SFA phonics lessons. 


A small number of teachers, four out of 24, felt The Electric Company content confused 
students. While five teachers felt the content of The Electric Company was not 
appropriate for children in their class. Two teachers expressed concern that the 
character of the “Butcher” was too violent and inappropriate for their students. 


The Electric Company School’s Initiative Curriculum 

Findings from the online survey show that 22 teachers found it “easy to build bridges, 
between The Electric Company content and the rest of the Success for All Curriculum.” 
In addition, 21 out of the 24 teachers reported that they used the language/songs from 
The Electric Company School’s Initiative Curriculum when they were teaching phonics 
concepts during the rest of the SFA Curriculum. These findings suggest that teachers 
were engaged with The Electric Company School’s Initiative Curriculum enough to make 
connections between TEC content and the rest of the SFA Curriculum. 


Teachers were asked to compare the SFA Lessons with The Electric Company content to 
the traditional SFA lessons on factors of student engagement, the length of time to 
deliver the lesson and their own interest. Findings suggest that at least 70% percent of 
teachers felt they witnessed “More than usual” student engagement in the areas of 
student participation; attention; and laughter. Additionally, teachers felt their own 
interest also increased “More than usual” when compared to the traditional SFA 
lessons. Our findings suggest that the length of time it took to teach The Electric 
Company School’s Initiative Curriculum lessons remained about the “Same as usual” 
according to 11 teachers, but was deemed “More than usual” for about eight. The 
following figure (15) summarizes the findings on how teachers felt The Electric 
Company School’s Initiative Curriculum compared to the traditional SFA lesson on the 
previous factors indicated. 
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Figure 15: How did the SFA lessons with The Electric Company content compare to the 
traditional SFA lessons. 


How did the SFA Lessons with The Electric Company content compare to the 
traditional SFA lessons on the following factors: 
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Responses about the technology 
23 teachers delivering The Electric Company School’s Initiative Curriculum deemed the 
interactive white board essential to delivering the curriculum. 


Teachers’ responses to the video segments 
Further findings regarding the use of The Electric Company videos show that out of the 
24 teachers who responded, most agreed or strongly agreed with: 

* The pacing of The Electric Company content (20 teachers) 

¢ The amount of text-on-screen in The Electric Company video segments (22 


teachers) 

¢ The word choice used in The Electric Company videos was appropriate (21 
teachers) 

¢ That students constantly interacted with The Electric Company videos (23 
teachers) 


¢ The number of videos clustered around each phonics goal (23 teachers) 


The following figure (16) summarizes the findings on the teachers’ degree of agreement 
for each of the statements about The Electric Company videos. 
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Figure 16: Teacher’s agreement with statements about The Electric Company games. 


Please rate your degree of agreement for each of the following statements about The 
Electric Company Games. 
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Researchers asked teachers to select the top three Electric Company Videos from a list 
of TEC videos shown in class that elicited the most positive response from students. 
Researchers included screenshots of The Electric Company videos in the survey to 
enhance teachers’ recall of the episodes. Our findings suggest that “Silhouette,” 
“Anthems,” and “Music Man” elicited the most positive response from students. Figure 
17 summarizes the findings for the rest of The Electric Company videos. 
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Figure 17: Teacher selection of top three Electric Company videos. 


Select the top 3 Electric Company Videos from the following list, that elicited the most 
positive response from the students: 
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Teachers’ responses to the games 
Our findings regarding teacher’s experience with The Electric Company games show that 
out of the 24 teachers who responded: 
* 21 agreed the games enhanced the SFA lessons. Additionally, 
19 teachers agreed that instructions for the games were clear 
22 noted that students responded positively to the game’s user feedback 
20 teachers noted that vocabulary used in the games was at a right language 
level for the students 
¢ 23 agreed that the games conveyed the goals addressed by “Quick Erase,” 
previously used SFA content and replaced by The Electric Company games 


* 
* 


* 


Findings show that teachers were not as content with the pacing of The Electric 
Company games for their students. The following chart summarizes the findings on the 
teacher’s degree of agreement with each of the statements about The Electric Company 
games explained above (see figure 18). 
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Figure 18: Teacher rating of agreement with statements about The Electric Company. 


Please rate your degree of agreement for each of the following statements about The 
Electric Company Games. 
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Researchers asked teachers to select the top three Electric Company Games from a list 
of games used in the class that elicited the most positive response from students. 
Researchers provided participants with screenshots of The Electric Company games to 
enhance teachers’ recall of the games. “Word Transformer,” “Say What,” “Sort It Out,” 
and the “Chain Game” were selected as the top four games that elicited the most 
positive response from students, in rank order from most to least positive response. 
Findings from the online survey suggest that “Combotronic” and “Great Escapes” were 
the least favorite with students as perceived by teachers. Focus group interviews with 
The Electric Company/Success for All teachers indicate that “Combotronic” caused the 
greatest amount of technical difficulties. These findings suggest that technical difficulty 
most likely contributed to the student’s negative responses to these games. The 
following figure displays the findings on The Electric Company games that elicited the 
most positive response from the students (see Figure 19). 
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Figure 19: Teacher’s top three choices of Electric Company games that elicited the most 
positive responses. 


Select the top 3 Electric Company Games from the following list, that elicited the most 
positive response from the students: 


Number of Responses 


Word Transformer Combotronic Great Escapes Chain Game Say What Sort it Out 


To understand how students interacted with The Electric Company games, teachers 
were asked to indicate how students interacted with the games portion of The Electric 
Company School’s Initiative Curriculum. Findings show that teachers generally allowed 
students to work in pairs, or groups, and alternated between students communicating 
their response to the teacher and using the interactive white board themselves for 
submitting their answer. The following summarizes the findings on teachers and 
students interaction with The Electric Company games (see Figure 20). 
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Figure 20: Teacher responses to how students participate in game play. 


How did students participate in game play? (Check all that apply) 
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Using the Activity sheets 


Findings show that 16 out of 24 teachers used The Electric Company activity sheets. The 
following figure summarizes the teachers’ use of the activity sheets (see Figure 21). 
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Figure 21: Teacher’s agreement with statements about The Electric Company activity 
sheets. 


Please rate your degree of agreement for each of the following statements about The 
Electric Company Activity Sheets that were available to you as part of the lessons. 
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Using the Digital Literacy Resource 

Findings from the online survey confirm that 6 out of the 24 teachers used the Digital 
Literary Resource (DLR). Out of the six teachers who used the DLR, two teachers 
indicated that they used it less than 15 minutes per week, and four teachers indicated 
that they spent about 15-30 minutes per week on the DLR site. When asked to report 
how they used the DLR, teachers indicated that they used it more as a “Lesson 
enrichment” than as “an assessment tool.” The following figure indicates teachers’ 
degree of agreement for each of the following statements about the DLR (see Figure 
22). 
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Figure 22: Teacher’s agreement with statements about the Digital Literacy Resource. 


Please rate your degree of agreement for each of the following statements about The 
Digital Literacy Resource (DLR). 
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Future use 

Our findings in speaking with teachers and through surveys, both from multiple-choice 
questions and open-ended responses, show that teachers would be excited about using 
more SFA lessons with The Electric Company content in the future. Almost all surveyed 
teachers would look forward to delivering more The Electric Company/Success for All 
lessons and materials. In accordance with the online survey responses, only two 
teachers would have rather continued “business as usual, instead of using The Electric 
Company content in the SFA lessons.” 


Teachers were asked whether they would ever screen whole episodes of The Electric 
Company for their class. Findings suggest that 15 of 20 teachers who responded are 
willing to present whole episodes in class. After speaking with teachers during a site 
visit, teachers confirmed they would be interested in screening whole episodes, but 
indicated that they had difficulty in accessing shows online. Furthermore, teachers 
expressed a strong concern about time constraints. One teacher indicated the following: 


| would as an end of reading activity when we have time leftover after covering 
the curriculum. However, | would NOT replace the existing SFA Curriculum 
aspects with Electric Company, as the kids benefit so much from the SFA 
material. 
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Three teachers also mentioned that screening episodes would help students become 
familiar with The Electric Company characters. One teacher stated the following: 


In SFA we have phonics assessments about every two weeks. During this time 
students are working independently or with their partners. During one of my 
assessments | actually went online and downloaded one of the Electric co. 
episodes so that the students could get familiar with the characters. They loved it 
so much they keep asking for more. 


Researchers coded the responses to the open-ended questions to find the most 
common responses across all teachers’ submissions for each specific question. The 
following are the five open-ended questions and a summary of the findings. 


What did you like best about The Electric Company parts of the lessons? 


Findings from the open-ended responses show that 15 teachers identified student 
engagement and interaction with TEC parts of the lessons as the most valuable 
enhancement. One teacher mentioned that her “kids are more interested in learning, 
the repetition of the sounds helps them learn better and helps make them be more 
successful at reading.” The same teacher acknowledged, “the kids really love them and 
they are so into it that it makes teaching phonics a lot of fun.” In addition, teachers 
mentioned videos as a more significant contribution to the lessons as compared to the 
TEC games, and noted that The Electric Company parts of the lessons were a positive 
contribution to the structure Fast Track Phonics lessons curriculum. 


What did you like least about The Electric Company parts of the lessons? 


Nine teachers reported that The Electric Company parts of the lessons were sometimes 
not compatible with the grade level they were teaching and that they were inconsistent 
with the SFA phonics lessons. Furthermore, seven teachers felt the pace of the lessons 
needed some adjustment and the activities’ (games) duration could be adjusted to 
better fit their needs. Five different teachers reported technical challenges and 
difficulties with graphics/sound. A teacher explained the following: 


Sometimes the smart boards or program wouldn't function well. We wasted 
some time trying to deal with technology problems and regaining students’ 
attention once things went back to normal. Also, some games were kind of long 
and would lose some students’ interest in the mean time. At the beginning, 
students seemed uninterested in participating with the first review lessons 
because they were already familiar with the content but then once the other 
games, songs, and videos started coming up, students seemed to gain more 
interest and enjoyed themselves participating. 


How could The Electric Company part of the lessons be improved? 
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Six teachers would like to see improvement in game play and interaction with games 
with regards to pacing, difficulty, functionality (frozen screen) and clicking/dragging. 
One teacher suggested the following, 


| like to speed up other portions of the games. For example when the students get 
a word correct, the celebration or acknowledgement takes too long. | see the 
purpose but a stronger shorter celebration would be just as impacting. 


In addition, four teachers felt the graphics could be improved by allowing the user to 
zoom in/out within windows and allow for sound (volume) control. As previously 
mentioned in what teachers liked least about The Electric Company parts of the lessons, 
teachers would like to see a better integration with SFA lesson. Through either a more 
comprehensive integration into the curriculum or simply more content to allow teachers 
to choose what lessons best fit their student’s literacy needs. 


Why or why not, do you believe students benefitted from participating in the project? 


Twenty-two out of 23 teachers felt their students benefitted from participating in the 
project. Perceived student engagement and general improvement to the lesson’s 
structure was the most common response as to why! teachers feel their students 
benefited from their participation. While two teachers saw improvement in their 
students’ reading scores, four teachers noted that the curriculum was mostly used as a 
review of phonics lessons. One teacher described her positive experience with the 
curriculum as the following: 


The lessons we taught were a needed review for struggling readers. They literally 
begged to play the games again and again. And the video clips were sung on the 
way out of the door (silent e is a ninja....) That convinced me that TEC is a more 
modern format possibly more pleasing to this generation. Many told me that 
they were watching the show at home after school. 


Is there anything else you'd like to tell us about your experience using the program? 


Teachers are very eager to receive more of The Electric Company material, and would 
like to deliver a more comprehensive The Electric Company School’s Initiative 
Curriculum at the start of the school year. One teacher stated: 


| completely love it because it has saved us tons of time, engaged the students in 
a positive manner, and improved our reading scores overall. The students love to 
do phonics and pay attention because they know they are going to play a game. 
They also watch the videos carefully and associate with the sound that they have 
learned. It has made learning to read a lot more fun for the students. 


102 


lam very happy with the program. The SFA lessons are much more visual for the 
students and they keep us on task. Everything is provided and that is a very 
positive for the students and the teachers. Teachers have more time to teach. The 
games are also excellent because they are not boring for those students that 
need the extra stimulation and motivation. Thank you for all your work and 
commitment to the SFA program and to our school. Our children need this type of 
stimulation and motivation. 
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Appendix E 


Interview Summaries 


Interviews were conducted with teachers, camp facilitators, parents and producers and 
staff from production companies funded through the Technological Enhancements 
project. Nine interviews took place and included 20 individuals in one-on-one or group 
interview settings. The number of interviewees was not consistent across properties, 
and the number of interviewees in each category: teachers, parents, and 
producers/staff, were quite small. Therefore the use of interview materials in this report 
is limited to providing context, raising issues for further examination through other data, 
and confirming issues identified through other data. Despite the limitation of size of the 
interviewee sample, the interviews shed light on several topics that are considered 
important or relevant to the development of digital tools that seek to support literacy 
skills among young children and the adults who care for them. 


The interviews were designed to draw out information from different stakeholders 
about the development and implementation of the three properties, with an emphasis 
on the addition of the technological enhancement. Producers at each of the properties 
were asked to discuss the change between previous products and current ones, WHY! 
specific choices were made regarding the use of technology, what needs the new 
product addressed, and how the technologically enhanced property was viewed by its 
target audience. 


A total of 849 interview statements were coded into 14 categories. The largest group of 
interviewees was producers, with a total of 11 included in either one-on-one or group 
interviews. Five teachers were interviewed representing TEC School’s Initiative 
Curriculum and WordWorld eBooks, and four parents were interviewed representing 
WordWorld eBooks. Below we summarize interview comments by interviewee and by 
central themes that emerged. 


Producer and property staff interviews: 


Producer and staff interviews generated 474 coded responses. The majority of these 
statements addressed aspects of technology use, product development, the needs and 
skills of the end users. Producers and staff made only one comment about parents and 
only two statements were made about the provision of technology support or help to 
sites or individuals. 


When discussing technology use, producers and their staff talked about how they came 
to decisions about developing the technological enhancements: 
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...we did focus groups in all three locations and then we also did observations... 
what we found out really was that teachers loved the idea. - The Electric 
Company Producer 


Basically it’s another format through which the pedagogical goal of the TV show 
[is addressed] but done in a more interactive online way that can be 
disseminated. — WordWorld eBook Producer 


They also talked about the steps they took to inform the development process: 


We went to a preschool... and the teacher ran the [program] there and we just 
sat back and observed and watched the kids play different games and evaluated 
how they were playing and how they were playing them in groups and how they 
were playing them in relation to the curriculum that was being taught and how it 
was fitting in with the day. If it was running smoothly, if it wasn’t running 
smoothly. — Super WHY! Producer 


On this particular project... the content that we created was based on... all of the 
national reading assessment literature, so we knew we had good content. It was 
just trying to figure out how that matched in with what the teachers were 
doing... - The Electric Company Producer 


When discussing the needs of the end user they tended to talk about how the their tools 
supported specific skills: 


.. you're looking for some kind of juicy video content or clips or games to help 
reinforce those skills. You type in CH, it gives you a list of the things you can use. 
You can put them in a folder to save for later. You can download them onto your 
computer. They’re supposed to be ultra flexible since we know that what 
teachers are really short on is time and, of course, energy all of the time. —-The 
Electric Company/Success for All Producer 


Because one of the things that you’re aiming for is “stickiness.” Not only do we 
[provide repetition] for pedagogical purposes within the [product], the word 
sounds, the (phonemes) etcetera, we know that children will come back... time 
and time again and it drives the lessons home. — WordWorld eBook Producer 


Teacher Interviews: 


Teachers’ frequency of comments about parents and caregivers, and about the 
provision of technology support, mirror that of the producers. This group made only two 
comments regarding technology support or help, and four about parents or caregivers. 
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Five teachers were interviewed either in pairs of two or individually and a total of 329 
comments were coded for this analysis. Teachers most often discussed their own or 
their student’s use of technology in their classrooms: 


In the beginning you want to show them, you want to model for them how to log 
on, which levels might be appropriate for them at that given time. As they 
progress you really want to give them that power for themselves to choose which 
activities, what level they need to be working on. Because at some point, they 
know. And they want to have their freedom to choose the activity that they do 


for the day on the computer and the level. — Super WHY! Summer Reading Camp 
Teacher 


| just feel like if they like it at home and then we can do it at school, then maybe 
they'll look forward to reading class and get to see what video they see that day. 
— The Electric Company/Success for All Teacher 


The teachers were next most likely to discuss their response to the program within the 
context of their teaching environment. These responses included information about the 
technology (video, white board) and also about the content and its appeal for their 
students [example]. 


And all four of my ESEs [Exceptional Student Education] are either speech or 
language delayed and developmentally delayed. They enjoy the computer game 
because they can kind of pick what they want to do. They can put the 
headphones on and they’re in their own world, they can work at their own level. | 
like the beginner, medium, hard, because they can pick their own. — Super WHY! 
Summer Reading Camp Teacher 


| think it’s great for the reviewing of sounds or introducing the various sounds, 
because that’s the whole thing. — The Electric Company/Success for All Teacher 


Teachers generally responded positively to the technological enhancements and found 


the programs they were implementing to be engaging, educational and to be a positive 
addition to their instruction activities. 


| loved how the games extended the skill of the day for each character. | loved 
the stimuli. They were colorful, there was zipping, and popping. The kids loved 
the way they were able to interact with each character and sing their special 
songs especially with Wonder Red (all-ball- call...... so catchy!) — Super WHY! 
Summer Reading Camp Teacher 


The students responded very positively to the videos. The students also enjoyed 


the games. It's funny to hear some of the students memorize the lines of some of 
the characters in the videos. | just recently finished all The Electric Company 
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videos and | have been teaching the phonics without The Electric Company 
segments. The kids are so disappointed that there aren't any more electric 
company videos/games in our lesson 7 phonics. | have been playing the DVD that 
was given to us just to review certain sounds. The kids really love them and they 
are so into it that it makes teaching phonics a lot of fun. — The Electric 
Company/Success for All Teacher 


However they also identified challenges they faced when implementing the programs in 
terms of the technology: 


The Word Transformer. There’s just a lot of glitches still in the program. Like, we 
were talking, the Word Transformer and Silent E, | kept a little log or whatever 
just to remind myself, and the only time that the Word Transformer changed 
from Silent E, was Lesson 16 and it was Bossy R. Or the Chain Game, it would 
stick and you’d have to start over. - The Electric Company/Success for All Teacher 


They also identified challenges with the content: 


| found it difficult with the spelling. Because they’re four and five. | mean they’re 
turning five today. The Alpha Pig, that’s the game that they’re most successful at. 
It’s the letter recognition and the sound recognition. That’s pre-K appropriate. | 
felt like the spelling of the words, that’s a little far above. That’s probably first, 
second grade. First grade level, not pre-K. But they like the graphics. The graphics 
are awesome. The graphics are very nice. | think that’s what keeps them 
interested too. - Super WHY! Summer Camp Teacher 


During the one group interview conducted with four parents, parents tended to discuss 
the product predominantly in terms of their child, and also mentioned their own use or 
knowledge of technology, For example, one parent was familiar with whiteboards ,“I 
knew about the whiteboard cause I work at the schools that had it.” Parents did not 
address many of the themes discussed by producers and teachers including production 
issues, the role of the products in the future, responses of teachers, or the need for 
technology support. 


My grandbaby used to be lazy about reading. Since she’s in the program now she 
brings all kind of books home to read. - The Electric Company/Success for All 
Parent 


And she runs at home, jumps on the computer at home and the first thing she’ll 
go to is... | don’t know how to do it, they get on there and she’s pronouncing all 
the words and stuff she has on the computer. - The Electric Company/Success for 
All Parent 
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Appendix F 


Super WHY! Summer Reading Camp Facilitator Online Survey 


Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


1. Information and Assent 


* 1. This survey is designed to collect information from teachers who have taught in the 
2010 Super WHY Summer Reading Camps and would like to share their thoughts and 
experiences regarding the addition of the web-based computer games to the 
curriculum. The integration of web-based games to the Super Why Summer Summer 
Reading Camp curriculum Is an innovative approach to supporting aspects of carly 
iteracy, and as implementers of this curriculum you are able to provkte the program 
with insight into how it has worked for you and your campers. 


Researchers at the Education Development Center are conducting this evaluation to 
determine how to best optimize the web-based coment in the Super WHY Summer 
Reading Camps and we would like to hear from you about your use of the computer 
games in the camp. The information we collect with this survey will be used to support 
Gevelopment of this curriculum to improve reading Instruction for young children. 


Any information you provide on this survey will be kept confidential, and all information 
collected in this survey will be stored without any personal identification information. 


By checking the box below you indicate that you have read the information provided 
above and agree to complete the survey for the purpose of this study. Should you 
choose to discontinue your participation in ihe study, you can withdraw at any time, 
However, any information that you have provided to that point will remain part of the 
study and may be used in later analysis and reports. You may also decide not to answer 
any questions asked during the survey without prejudice. If you have any questions or 
concems regarding this study, please teel free to call or email Naomi Hupert at 707-829- 
8532 (nhupert® ede.org). 


O Fee | Pe ETO Ma Ohl BD ove OFC) bg oe to Ge Digete 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


2. General information 


* 1. Have you used computers or computer games with students in your classroom In the 


2. Please select the age of the children you taught this year in the Super WHY Summer 
Reading Camp. Check all that apply. 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


3. Using the Computer Games 


* 1. Please rate your degree of agreement for each of the following statements about the 
Super WHY web-based computer games and the Super WHY Summer Reading Camp 
Curriculum, 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


4. Timing and Supervision 
* 1. Please answer Yes or No to the following. 


Od te ertedee of ertetes eree ting me oteceor of the compte germet ent fer paw coTony” 


O 
O 
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* 2. On average, how many adult supervisors (including you) were present during the day 
to help the campers? 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


5. Computer Games 


* 1. Please rate your degree of agreement for each of the following statements about the 
Super WHY web-based computer games. 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


6. Computer Games 


Use these images to help you anewer the next two questions. 


* 1. Please select the games your campers played. (Check all that apply) 
[L] were rere tonne enve te yong 
[] Prncese Proms Gon Conan Spating Boe 
[FW Fw aeons 
— 


* 2. From the list below, choose the two (2) Super WHY computer games that elictied the 
most postive response from the students: 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


7. Camper Participation 


* 1. How did campers participate in computer game play? Check alt that apply. 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 
8. 


* 1_ ts this the first time you have facilitated a Super WHY Summer Reading Camp? 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 
9, Reflections 


* 1. How did the Super WHY Summer Reading Camp curriculum with the addition of the 
computer garnes compare to the requiar Super WHY Summer Reading Camp curriculurn 
without the computer games on the following factors: 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 
10. Reflections 


* 1. What did you like best about the computer games? Please Explain. 
| 
2 
* 2. What did you like least about the computer games? How could this be improved? 
“| 
L 2] 
* 3. Do you believe students benefited from using Ihe Super WHY computer games? 


4, ts there anything else you'd like to tell us about your experience using the computer 
games? 


| 
J 
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Facilitator Survey: Super WHY Summer Reading Camp and Web-based 


11. 


1. Thank you for taking the time to participate in this survey. The information you 
provided will help us with our evaluation to determine how to best optimize the web- 
based content in the Super WHY Summer Reading Camps. We would like to send you a 
$20 gift certificate as a thank you. in order to do this we need to have your full name and 
email address. Please provide for us below. 


i've Yet 
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2. No thanks, 
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The Electric Company School’s Initiative Curriculum Teacher Online Survey 


Teacher Survey: TEC/SFA Curriculum 


1. information and Assent 


* 1. This survey is designed to collect information trom teachers who have worked with 
the Success For All (SFA) /Electric Company curriculum and would like to share their 
thoughts and experiences regarding the use of this curriculum. The SFA/ Electric 
Company curriculum is an innovative approach to supporiing aspects of elementary 
iteracy and as implementers of this curriculum you are able to provide the program with 
Insight info how it has worked for you and your students. 


Researchers at the Education Development Center are conducting an evaiuation of this 
curriculum and would like to hear from you about use of the Interactive Whiteboard, 
Video segments and Digital Library Resource (DLR) components of this curriculum. The 
information we collect with this survey will be used to support further development of 
thés curriculum, and others like it that seek to improve reading insiruction for young 
children. 


Any information you provide on this survey will be kept confkiential, and all information 
collected tn this survey will be stored without any personal identification information. 


By checking the box below you indicate that you have read the Information provided 
above and agree to complete the survey for the purpose of this study. Should you 
choose to discontinue your participation in the siudy, you can withdraw at any time. 
However, any information that you have provided to that point will remain part of the 
study and may be used in later analysis and reports. You may also decide not to answer 
any questions asked during the survey without prejudice, If you have any questions or 
concerms regarding this study, please feel free to call or email Naomi Hupert at 707-829- 
8532 (nhupert @ ede.org). 
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Teacher Survey: TEC/SFA Curriculum 
2. Implementation Background 


1. Mave you used any of the following digital media materials prior to using the SFA/ 
Electric Company curriculum (Not necessarily The Electric Company curriculum)? 


Nol we inmeteres O-eprerty 


2. Please select the general reading level of students who participated in the SFA/ 
Electric Company curriculum. 


3. Please select the number of the SFA/ Electric Company curriculum lessons you 
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Teacher Survey: TEC/SFA Curriculum 


3. Using the SFA/Electric Company Curriculum 


1. Please rate your degree of agreement for each of the following statements about the 
SFA/ Electric Company curriculum, 
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2. Please rate your degree of agreement for each of the following statements about 
Electric Company Vikieos, 


7 
. 
. 
ay 
> 


O0000} 


i 


Te peamg of The Cie (orpety €48 apgr”e ate fo %y MLGeves 

Tere o08 © mehoert erout of teston ereen @ The Deore Compe vetes 

we gpreris 

Pee or Ge weed me "Re Lieut Come ety veheed ere of a8 Agere ate bevel lo re 
ete 

Te tems consent) eteected (eg cet e94 eng erte eeee et gees He 
vubes pewed laughed) ett Thee f henty oe) vos 


Tie St Of Ont Catered & Cutt C80) PRowere goa ees AM OMe 


00000! 
00000; 
00000 


The next question refers to the following video segments. 
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Odd Couple 
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Teacher Survey: TEC/SFA Curriculum 


* 3. Select the top 3 Electric Company Videos from the following list, that elicited the most 
positive response from the students: 
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4. Did you use any of the language/songs from the SFA /Electric Company curriculum 
while you were teaching phonics concepts? 


5. Please rate your degree of agreement for each of the following statements about The 
Electric Company Games. 
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Teacher Survey: TEC/SFA Curriculum 


* 6. Select the top 3 Electric Company Games from the following list, that elicited ihe most 


We 
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Teacher Survey: TEC/SFA Curriculum 
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Teacher Survey: TEC/SFA Curriculum 


4. The Electric Company Activity Sheets 


1. Did you use The Electric Company Activity Sheets? 
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Teacher Survey: TEC/SFA Curriculum 
5. The Electric Company Activity Sheets (cont.) 


1. Please rate your degree of agreement for each of the following statements about The 
Electric Company Activity Sheets that were available to you a8 part of the lessons. 
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Teacher Survey: TEC/SFA Curriculum 
6. Digital Literacy Resource 


1. Did you use the Digital Literacy Resource (DLA), searchable database of all The 
Electric Company-related coment by curricular goal? 


O-» 
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Teacher Survey: TEC/SFA Curriculum 


7. Digital Literacy Resource (cont.) 


1. Please rate your degree of agreement for each of the following statements about The 
Digtial Lheracy Resource (DLA). 
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2. How many minutes per week did you devote to the Digital Literacy Resource? 
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3. How did you use the interactive games in the Digital Literacy Resource? 
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Teacher Survey: TEC/SFA Curriculum 
8. Reflecting on the SFA/Electric Company Curriculum 


1. How did the SFA Lessons with The Electric Company content compare to the 
traditional SFA lessons on the following factors: 


Seth oe © pe Dap ane 
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2. What did you like best about The Electric Company parts of the lessons? 
a 
a 

3. What did you like least about The Electric Company parts of the lessons? 
| 
aj 


4. How could The Electric Company part of the lessons be improved? 
=| 
=) 


5. Have you watched full episode(s) of The Electric Company? 


6. Would you ever screen whole episodes of The Electric Company for your class (28 
minutes running time)? 
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Teacher Survey: TEC/SFA Curriculum 


7. Would you be interested in using The Electric Company content to teach vocabulary 
and reading fluency as well? 


I 
J 
9. ts there anything else youd like to tell us about your experience using the program? 
| 
si 
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Teacher Survey: TEC/SFA Curriculum 


9. Survey Completed! 


in order to recetwe your $20 of! certificate from Amazon com, please prowde us with the fofiowing information 


* 1. Firat Name: 
* 2 Last Name: 


* 3. Email: 
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